Memory leaks in computer programing.

Debio

Back from the land of the dead.
Super Member
Registered
Joined
Jan 16, 2012
Messages
1,526
Reaction score
217
Location
Japan
I am very basically familiar with memory leaks in low level computer languages like C.

I can do hello world stuff, if I have the book with me. More complicated stuff with a book and examples. Things like that.

I have an idea for my WIP that will hinge on a memory leak situation, that is if my basic understanding is not totally incorrect. So to the question.

A memory leak occurs when a program claims a block of memory for some action and for some reason fails to release that block of memory when it is finished using it. So the next time it wants to perform that action it will claim a block of memory to do it.

Since it is not releasing these blocks when it is done using them, eventually the computer will run out of memory and bad things happen, like the computer slows or possibly even crashes.

To fix this problem, someone with access to the source code needs to go in and find the particular function that is not releasing the memory and fix that code.

Without access to the code, that can't be done.

If the system is rebooted, will that fix all the symptoms of the memory leak and return the system to smooth operation? At least until the memory is all used up again?

I'm pretty sure the answer is yes, I just want to make sure so I don't look like a total idiot if someone who knows their stuff reads the story.
 

alleycat

Still around
Kind Benefactor
Super Member
Registered
Joined
Apr 18, 2005
Messages
72,873
Reaction score
12,224
Location
Tennessee
Yes, a complete reboot would solve the problem in this case (there could be problems were it wouldn't). I suspect most people would take some action when they noticed that their computer was not performing as usual. For example only, they might bring up Task Manager if they were using Windows and kill the processes.
 

Anaximander

Super Member
Registered
Joined
Feb 11, 2010
Messages
293
Reaction score
57
Location
Evesham, UK
That's basically it. Some languages have a system in place known as a "garbage collector" which can detect when a particular variable is no longer in use, and will automatically free up the memory where that variable is stored (C# and Java are two well-known examples) but many languages don't do this - particularly the older ones. C and C++ don't, and they're VERY widely used.

If a system is experiencing a memory leak, a reboot will free ALL memory and fix the problem temporarily. The next time the problem program is launched, the problem will return. The rate at which the memory leak fills memory will be the same, assuming the program is behaving the same - if the memory leak is caused by a function that the user triggers, then obviously you'll run out of memory faster if you do that action a lot, but if it's something that just happens periodically in the program's running then it'll happen at about the same rate each time. If you don't reboot, you'll run out of memory (RAM, that is). Your system will get slow as it gets to the last of its memory, and will probably then stop responding properly, or at all. Some operating systems handle memory leaks better than others, but generally (especially with Windows, which is what most people will be familiar with) that's how it'll go. Often the easiest way to fix this is to just kill the power - hold down the power button until the screen goes black. If the problem program is closed before you run out of RAM, your system will leave that block allocated, so it won't get any better, but it won't get any worse either. Re-launching the problem program will continue from where it left off, getting gradually worse again until you run out of RAM.

The fix is to go through the code and find out where you're not freeing up your memory (in C++, this is generally an issue with dynamic allocation, which is where you allocate stuff on the heap, not the stack... it's a little complex to explain if you're not a programmer or computer scientist, but you can get by without knowing it; if you find yourself needing details, there should be enough here for you to Google :p ).

Without access to the source code, you're stuck with periodic crashes, or having to reboot regularly. This means you'll need to get in touch with the developer - or, if it's open source and you're a good programmer, go download the source code and have a look yourself.
 

Mac H.

Board Visitor
Super Member
Registered
Joined
Feb 16, 2005
Messages
2,812
Reaction score
406
You are right - although you can usually just kill the program to eliminate it as well ... as long as the program didn't try to share the space with someone else the OS should kill it when the program exits.

One thing to keep in mind is that with modern languages memory leaks are much rarer - you have garbage collection and smart pointers that automatically keeps track of it.

A managed code enthusiast would argue that it is impossible to leak memory in a modern language like C# .. but it is still possible, particularly with subscribing to events.

---

The reason I mention it is because your description sounds like how a programmer from the ancient past of the 1990s (like me - !sob!) would describe the problem ... but a more modern description would be about "Leaving a hanging subscription, so the garbage collector never destroys the object."

Mac
 

sciencewarrior

It's alive!
Super Member
Registered
Joined
Jul 14, 2012
Messages
719
Reaction score
71
In theory, someone with sufficient knowledge of low-level code can alter the program to fix the bug. It involves changing the calls to the problematic function to another function that they create, which fixes the problem. That's one of the techniques pirates use to crack games with DRM.

With modern hardware and software, you rarely have to reboot the whole machine to recover from one misbehaved program. The OS has enough safeguards to keep a minimum of resources, and automatically finish programs that are taking too much. But in older machines, or special-purpose machines with limited resources, this kind of thing happens frequently.
 

Debio

Back from the land of the dead.
Super Member
Registered
Joined
Jan 16, 2012
Messages
1,526
Reaction score
217
Location
Japan
Thank you everyone. I am not a programmer, exactly, but I did study CS for a few years in the early 90s. Hence my ancient description of what I wanted.

I am glad to hear that a good OS will handle this well so that just rebooting the program will provide the temporary fix. That will help a lot.


Again thanks.
 

MoLoLu

Super Member
Registered
Joined
May 20, 2011
Messages
540
Reaction score
47
I'm not an expert, being someone who came into programming after garbage collection became common, but I can confirm what you say is correct.

As others have stated, modern languages with garbage collection solve much of the issue but there's one other to remember here: bad design.

If, for whatever reason (generally related to coding at two in the morning and not thinking straight), you're keeping objects alive longer than they are needed (i.e. referencing something which should be out of scope from somewhere which is still in scope), you can still end up with something that resembles a 'memory leak'. It isn't technically a leak because the memory is still 'in use' but, if you aren't actually using the object you referenced and it's just hanging in limbo, it results in a similar issue of keeping memory blocked for no sensible reason.
 

Debio

Back from the land of the dead.
Super Member
Registered
Joined
Jan 16, 2012
Messages
1,526
Reaction score
217
Location
Japan
In theory, someone with sufficient knowledge of low-level code can alter the program to fix the bug. It involves changing the calls to the problematic function to another function that they create, which fixes the problem. That's one of the techniques pirates use to crack games with DRM.

With modern hardware and software, you rarely have to reboot the whole machine to recover from one misbehaved program. The OS has enough safeguards to keep a minimum of resources, and automatically finish programs that are taking too much. But in older machines, or special-purpose machines with limited resources, this kind of thing happens frequently.

So, can a hacker add these functions without actually needing to access the actual code itself?

It seems like that should be possible, since there are things like Wine and DOSbox. Something they can insert between the OS and the problematic program to make a makeshift fix?
 

cbenoi1

Banned
Joined
Dec 30, 2008
Messages
5,038
Reaction score
977
Location
Canada
A running program with a memory leak will grow its process memory footprint until the OS swaps memory blocks in and out (also called thrashing). This is because the maximum process size usually is much bigger than the amount of physical memory (RAM) and thus the hard disk is used to page memory blocks in and out on a per-demand basis. The user will sense two things: 1) a sharp decline in overall system performance, and 2) a hard disk light turned on permanently.

The process will continue to grow until it reaches a pre-set process size limit at which point the OS will terminate the process. On a Windows system, the process simply disapear and an entry is made into the system logs. On a *nix system, there will be file created called a 'core dump'.



As others mentionned, you would find this problem on programs written in 'C' for Unix machines. So we're talking about the 1980 - 1990 era. 'C++' made memory management more bearable. Then you have Java and the more recent C# languages which do manage the memory pool on behalf of the application.

So a piece of software with a memory leak would be dated.



-cb
 
Last edited:

cbenoi1

Banned
Joined
Dec 30, 2008
Messages
5,038
Reaction score
977
Location
Canada
> So, can a hacker add these functions without
> actually needing to access the actual code itself?

Trying to fix an executable without the corresponding source code is extremely difficult. Size does matter. And memory leaks are typically the hardest bugs to find.

-cb
 
Last edited:

Al Stevens

Super Member
Registered
Joined
Mar 4, 2011
Messages
2,537
Reaction score
214
A programmer who develops code by using a contemporary integrated development environment will know at the end of a test session whether the program has allocated memory blocks that it hasn't freed. The debugger reports not only the address and size of the leaked block, but where in the program (which source code line number and file) the block was allocated. The most typical way a program would (could) be released with such bugs is if the programmers and testers have not tested all possible input conditions. That is, data get into the program in the field that no one anticipated during the tests.
 

Debio

Back from the land of the dead.
Super Member
Registered
Joined
Jan 16, 2012
Messages
1,526
Reaction score
217
Location
Japan
I'm not an expert, being someone who came into programming after garbage collection became common, but I can confirm what you say is correct.

As others have stated, modern languages with garbage collection solve much of the issue but there's one other to remember here: bad design.

If, for whatever reason (generally related to coding at two in the morning and not thinking straight), you're keeping objects alive longer than they are needed (i.e. referencing something which should be out of scope from somewhere which is still in scope), you can still end up with something that resembles a 'memory leak'. It isn't technically a leak because the memory is still 'in use' but, if you aren't actually using the object you referenced and it's just hanging in limbo, it results in a similar issue of keeping memory blocked for no sensible reason.

I was never very good at coding and my knowledge is 15 years out of date, so bear with me on this.

The problem I'm envisioning is very similar to your last paragraph.

There is a bit of memory that needs to be cleared up. It is sent to a function to do that. But for some reason, that function isn't working properly. There is an exception handling(proper term?) routine that does a makeshift stick it here for now kind of thing till it can be fixed properly. But the fix it later code was never implemented so it never actually gets fixed. Kind of like the code was written but mostly commented out. The function frame exists though.

Luckily this particular event is very rare, and the stick it here for now thing is not really noticeable and assumed by the users to be an intended part of the program. Until someone realizes that it is not supposed to happen this way and they realize there is something wrong with the code.

I really hope the above makes sense. I just don't want a that's not how it works reaction.
 

Debio

Back from the land of the dead.
Super Member
Registered
Joined
Jan 16, 2012
Messages
1,526
Reaction score
217
Location
Japan
This is for an SF/F story that takes place in a different time and place, so I'm looking mostly at conceptual level things. All of your answers have been very helpful in helping me focus this.

I'll be happy to explain in more detail what is happening story wise if you want. I just didn't want to do that because I thought it might confuse the issue more than it needed. But apparently coding practices have improved more than I had realized. I may have to rethink this a bit.

Or I could shift this to the brainstorming forum. What I thought may be a relatively easily question has given me more than I expected. This is good, the complications are making it more interesting for the story.
 

Deleted member 42

Even Windows has better handling of memory now.

You might want to read about more recent ideas like sandboxing.

There's also the issue that with more RAM memory leaks for individual apps are sort of a minor issue.
 

sciencewarrior

It's alive!
Super Member
Registered
Joined
Jul 14, 2012
Messages
719
Reaction score
71
I was never very good at coding and my knowledge is 15 years out of date, so bear with me on this.

The problem I'm envisioning is very similar to your last paragraph.

There is a bit of memory that needs to be cleared up. It is sent to a function to do that. But for some reason, that function isn't working properly. There is an exception handling(proper term?) routine that does a makeshift stick it here for now kind of thing till it can be fixed properly. But the fix it later code was never implemented so it never actually gets fixed. Kind of like the code was written but mostly commented out. The function frame exists though.

Luckily this particular event is very rare, and the stick it here for now thing is not really noticeable and assumed by the users to be an intended part of the program. Until someone realizes that it is not supposed to happen this way and they realize there is something wrong with the code.

I really hope the above makes sense. I just don't want a that's not how it works reaction.

It makes perfect sense, really. The program works fine for years and years, until the source code is lost, and every programmer who worked on it is retired. Then, one day, it crashes, for no apparent reason. The support personnel assume it's a fluke, check the inputs, restart the program, and a few hours or days later, it crashes again.

After the initial panic subsides, the tech team will probably set up some kind of watchdog to restart the program once it starts taking up too much memory. It doesn't really solve the problem, but it buys them some time.

So they set up a test environment and run the program. If they are lucky, that particular situation will also happen in the test environment. Otherwise, they won't have another option but to debug directly in production.

Two tools will be extremely useful. The first one is a good debugger. With it, they can stop the program at various points, examine the memory, and discover what data structures/objects are being kept in memory, instead of being disposed of. From there, they can trace what function isn't freeing them.

The second tool is a decompiler. It takes an executable and turns it back into source code. The translation isn't perfect; the comments disappear, the variables are likely named something like "var1, var2, var3", and some parts may look strange, even though they are still valid code. It's a bit like converting from English to Japanese and back, in that you lose a lot of the original information.

Even with these tools, it can still be a lot of trouble to find the defective function, specially if the program is complex, and does several things at once. But once it is found, fixing it and recompiling the program is relatively straightforward.
 

kuwisdelu

Revolutionize the World
Super Member
Registered
Joined
Sep 18, 2007
Messages
38,197
Reaction score
4,544
Location
The End of the World
Also, the seriousness of your memory leak will depend on the kind of app you're talking about. There are still plenty of applications that are coded in lower level languages like C and need to address enough memory to make leaks an issue.
 

F.E.

Sockpuppet
Banned
Joined
Jan 24, 2012
Messages
637
Reaction score
106
Memory leaks? An application program could have a problem like that, which would usually basically be "temporarily" fixed by restarting the program. (Remember, the smarter the pointers used in a program, then the bigger the hog in memory and execution time that program will be.)

If you tell us what you want your "computer problem" to cause for your story, then we might come up with other possibilities. :)

E.g. Database locks not available or held too long, which might also be involved in race conditions. Servers ported or upgraded to using a different flavor or version of an operating system (e.g. to Red Hat from a different Linux) and then newer built applications don't quite work as they used to. Different CPU/processor used (bug in one of cache levels) and would cause erratic errors. Ported to different hardware boards and race condition type of errors seen in database corruption--faulty bus locking. Etc. (I've been involved in at least one of each of those, and, er, actually caused one of them which wasn't identified for around a year later.)

Usually all the fault recovering code in the operating system is not actually, er, tested by being run through a test suite. Due to what it would actually take to test it, some of it might actually only be "visually" code reviewed. Though most of the bigger projects do have automated system testing with regression test buckets. But they're only as good as the scripts that are run, and there is a practical limit as to the number of scripts and the amount of time available to run them and the limited number of testbeds. And well, all that testing ain't going to be run everytime a minor software or hardware change is made. Er. Yeah.
 

F.E.

Sockpuppet
Banned
Joined
Jan 24, 2012
Messages
637
Reaction score
106
I was never very good at coding and my knowledge is 15 years out of date, so bear with me on this.

The problem I'm envisioning is very similar to your last paragraph.

There is a bit of memory that needs to be cleared up. It is sent to a function to do that. But for some reason, that function isn't working properly. There is an exception handling(proper term?) routine that does a makeshift stick it here for now kind of thing till it can be fixed properly. But the fix it later code was never implemented so it never actually gets fixed. Kind of like the code was written but mostly commented out. The function frame exists though.

Luckily this particular event is very rare, and the stick it here for now thing is not really noticeable and assumed by the users to be an intended part of the program. Until someone realizes that it is not supposed to happen this way and they realize there is something wrong with the code.

I really hope the above makes sense. I just don't want a that's not how it works reaction.
Yes, that is a real possibility, an exception handler with a block of conditional code for purely testing purposes that, by accident, hadn't been switched off, and so, the code is still generated in the final product. And since that code doesn't ever get executed during the normal test scripts (that unused code could be detected by an automated coverage tool) but can be waived due to the exceptional nature of the situation that has to be brought about in order to actually test it, that code gets signed off. Perhaps that test code can provide an unintended opportunity.
 

MoLoLu

Super Member
Registered
Joined
May 20, 2011
Messages
540
Reaction score
47
It makes perfect sense, really. The program works fine for years and years, until the source code is lost, and every programmer who worked on it is retired. Then, one day, it crashes, for no apparent reason. The support personnel assume it's a fluke, check the inputs, restart the program, and a few hours or days later, it crashes again.

Agree one hundred percent. Similar things have happened to me at work so many times due to minor errors which no one foresaw being a problem and suddenly everyone ending in a scramble of WTF why doesn't this works for weeks until someone takes the problem apart and finally figures out one line of code was wrong - usually the last place you look.

To be honest, modern systems have a lot of failsafes but one cannot, ever, account for human stupidity. I've personally written too much WTF code that ended up in production to believe that this is uncommon. I'm not being intentionally stupid but there are times I just want to get something done because people need a working tool and only realize I created a bigger problem days later when someone complains about minor issues.

I can remember one particular case where we created a script template to be used worldwide on several tens of thousand clients and no one (least of all me, who'd written the thing) realized that the logging didn't work on 64 bit machines. No one told me either, they just whipped up a workaround and kept using that, until it came up at a team meeting because we had an issue on a 64 bit client and, well, we fixed the official template. After four months of not logging on 64 bit clients. I felt incredibly stupid.

There is a bit of memory that needs to be cleared up. It is sent to a function to do that. But for some reason, that function isn't working properly. There is an exception handling(proper term?) routine that does a makeshift stick it here for now kind of thing till it can be fixed properly. But the fix it later code was never implemented so it never actually gets fixed. Kind of like the code was written but mostly commented out. The function frame exists though.

Luckily this particular event is very rare, and the stick it here for now thing is not really noticeable and assumed by the users to be an intended part of the program. Until someone realizes that it is not supposed to happen this way and they realize there is something wrong with the code.

I really hope the above makes sense. I just don't want a that's not how it works reaction.
If this happens in such a way that it can't be detected quickly (i.e. you have the minor memory issue which doesn't come up for years as you described) it's likely this will go unnoticed and cause one big headache. Quite an interesing premise for a story by the way. Sounds like my more interesting (frustrating and fun) days at work.

And, yes, such things in theory should be caught during peer reviews and tests but, let's be honest, mistakes still make their way into the best code because someone wasn't paying attention or was thinking about something else.
 

Debio

Back from the land of the dead.
Super Member
Registered
Joined
Jan 16, 2012
Messages
1,526
Reaction score
217
Location
Japan
Thank you all very much. I can give you a not so brief summary(700 words)of what is happening. It is essentially a memory leak, but not. It's weird. And I'm a tad nervous about going into detail.

Not because people will steal the idea. That wouldn't bother me at all.

I worry that I'll get the same reaction from real programmers that I have when an over the edge Anime geek starts asking me about Japan and Anime and the like. (I have no problem with people who are fans of Anime. I like it. It's the over the edge types.)
 

Al Stevens

Super Member
Registered
Joined
Mar 4, 2011
Messages
2,537
Reaction score
214
That's basically it. Some languages have a system in place known as a "garbage collector" which can detect when a particular variable is no longer in use, and will automatically free up the memory where that variable is stored (C# and Java are two well-known examples) but many languages don't do this - particularly the older ones. C and C++ don't, and they're VERY widely used.
C++ has the auto_ptr class, which implements a smart pointer that automatically deallocates memory assigned to it when the pointer's destructor function is called.
 

MoLoLu

Super Member
Registered
Joined
May 20, 2011
Messages
540
Reaction score
47
C++ has the auto_ptr class, which implements a smart pointer that automatically deallocates memory assigned to it when the pointer's destructor function is called.

If you don't forget to call the destructor, that is, so one doesn't quite end up at the same thing as with a garbage collector.

Given, even with garbage collection there's objects you'll have to manually destroy, but the safety net is still bigger than in the C/C++ world.

Edit: This post has been corrected by others more informed than me. See below.
 
Last edited:

cbenoi1

Banned
Joined
Dec 30, 2008
Messages
5,038
Reaction score
977
Location
Canada
> If you don't forget to call the destructor, that is,

Al is correct. auto_ptr will release the memory when it's destructor is called - which happens when the auto_ptr object goes out of scope.

Most of the memory leaks in C/C++ applications happen at the API interface level nowadays. Most OS-level API calls are still C-style and this creates a lot of potential confusion between the part of the code that seizes an OS resource and another part that's supposed to release it.


-cb
 

MoLoLu

Super Member
Registered
Joined
May 20, 2011
Messages
540
Reaction score
47
My bad. Not familliar enough with the C/C++ language. Learn something new every day.