Member Discussions

I've noticed on alot of muds (Actually all of them) that they have a
socket_free list aswell as a socket_list, when a character logs
out of the mud it takes it from the socket_list and places it within
the socket_free list, why do they do this as in the advantages and
disadvantages?

My guess would be faster because you do not need to allocate
memory but on the other hand your not freeing them directly?

Anyone wanna further my understanding please?

Thx
-leabear

Maybe malloc was slower back when these MUDs were written and it was an actual speed improvement. All I know for sure is that I recently ripped out ALL of those routines, got rid of SSM and got rid of DIKU/Merc/Envy's managed memory in my MUD. My MUD is now much, much faster then it used to be -- the players were actually able to tell I'd done something to it.

This is actually a small part of a bigger (and somewhat obsolete) issue.

Programs that manage large numbers of data objects tend to develop memory fragmentation when those data objects are allocated and deallocated haphazardly. Memory fragmentation amounts to 'gaps' in the program's address space, sections of memory that aren't in use and are too small to *be* used until other memory around it is freed. When you're dealing with tens of thousands of data objects, like in a MU*, these gaps can quickly add up.

The solution used by many MU*s is to allocate the data objects just once. When the data object is no longer needed, the program keeps track of it until it needs another data object of the same type, at which point the same memory space can be reused. Yes, you've sometimes got memory allocated that you aren't currently using, but odds are very high that you'll be reusing it in the near future anyway. (when a zone resets, or another player connects, or whatever)

Memory fragmentation isn't a problem for most programs, and RAM is so plentiful that space lost to fragmentation is often inconsequential on modern systems. But the original MU*s often had very little memory to work with, and couldn't afford to waste any.

Our Merc 2.1 based mud does not have such a construct, so I assume it appeared some time after 2.2 came out.

Memory fragmentation is one possibility, as the others have pointed out, another possibility I can think of is that it's there for retaining the file handles. You see as far as the operating system is concerned a socket is a file handle and it only has a certain number of file handles to share around. Once they run out, you can't open any other new files or create any other new sockets. So it would make sense for the mud to retain control of them to avoid other processes on the machine from stealing them and not giving them back. This is also why you have a fdReserve file handle that gets closed when you want to open a file and then reopened again immediately afterwards. It's to make sure that there's always a file handle available when you need it.

Genghis - this is great info; I didn't know that!

A question for you - like the memory aspect (where old machines had to worry because of shortages - and current machines altho they still need to worry don't need to worry as much) - is the same thing true about file handles? What is the finite number? How many are there? And does age of machine have anything to do with the amount?

Thanks

You see as far as the operating system is concerned a socket is a file handle and it only has a certain number of file handles to share around.

True

Once they run out, you can't open any other new files or create any other new sockets. So it would make sense for the mud to retain control of them to avoid other processes on the machine from stealing them and not giving them back.

Well, usually one would close the socket and place the data structure in a free-list, so the actual descriptor would be returned to the system, so I'm afraid that's not true (at least not in most MUD codebases).

I'm also not sure that the fdReserve trick works. There is nothing preventing another process on the machine from grabbing the descriptor just after we close fdReserve, not that I think we should care much about it, on an average linux machine, each user has 1024 descriptors available, and the server itself usually allows in the >10.000 range of total descriptors.

regards

Brian Graversen

1024 file descriptors per process, not per user. So fpReserve works, remove the descriptor reserve code and anyone can crash or disable your mud with a dos attack.

I still don't see why fpReserve should work. If the machine has used every single available descriptor, and you free fpReserve with the purpose of using it yourself, then what is the chance that one of the many other processes on the server won't use it before you do ? One available descriptor and several processes all wanting to get another free descriptor... besides, if someone was to take all descriptors on the server, the server would clog up anyway, and your MUD would still die.

Brian

fclose( fpReserve );
fpReserve = fopen( NULL_FILE, "r" );

fpReserve is only closed for a couple of cpu cycles before it's reopened to /dev/null or an actual file. Unless you're threading there's no way another function can 'steal' it.

>besides, if someone was to take all descriptors on the server, the server would clog up anyway, and your MUD would still die.

Log: New connect: 127.0.0.1 D6
Log: New connect: 127.0.0.1 D7
Log: New connect: 127.0.0.1 D8
Log: New connect: 127.0.0.1 D9
----
You patiently twiddle your thumbs.
----
Log: New connect: 127.0.0.1 D751
Log: New connect: 127.0.0.1 D752
Log: New connect: 127.0.0.1 D753
Log: New connect: 127.0.0.1 D754
Log: New connect: 127.0.0.1 D755
Log: New connect: 127.0.0.1 D756
Log: New connect: 127.0.0.1 D756
Log: New connect: 127.0.0.1 D756
----
Okay, now that's odd, let's have a look.
----
#Trying attempt 1 out of 20 attempts to connect.

#SESSION 'x' CONNECTED TO 'localhost' PORT '4321'

We have a limit of 750 players.
Please try back later.
----
OooOOooOoo, let's make a little code change just for you :)
----
installing........ [done]
Size of bin/emud (1) before stripping, (2) after stripping :
1.4M ../bin/emud
1.3M ../bin/emud
This compile gave :
- 0 warnings.
- 0 errors.
----
Here we go again
----
Log: New connect: 127.0.0.1 D6
Log: New connect: 127.0.0.1 D7
Log: New connect: 127.0.0.1 D8
Log: New connect: 127.0.0.1 D9
----
Cross-eyed and smiling as I watch the descs go twisting by.
----
Log: New connect: 127.0.0.1 D1020
Log: New connect: 127.0.0.1 D1021
Log: New connect: 127.0.0.1 D1022
Log: New connect: 127.0.0.1 D1023
----
Okie, interesting, let's have another look
----
#ses x localhost 4321
#Trying attempt 1 out of 20 attempts to connect.

#SESSION 'x' CONNECTED TO 'localhost' PORT '4321'

#SESSION 'x' DIED.
#SESSION '1' ACTIVATED.
----
Hrm, no error message on the client side, just disconnects. Too lazy to check that out.
----

Section                           Time (usec)    Freq (msec)  %Prog         %CPU
Update Time:                      3097          63206          0.08         0.00
Update Areas:                       30          69460          0.00         0.00
Update Weather:                     54          64016          0.00         0.00
Update Objects:                    523          68456          0.01         0.00
Char Save Time:                    119           6067          0.04         0.00
Update Violence:                  1003           4045          0.50         0.02
Update Obj Progs:                   27           4045          0.01         0.00
Update Mob Progs:                 1766           4045          0.88         0.04
Update Mobiles:                   1679           2022          1.69         0.08
Update Aggressive:                  83            505          0.34         0.02
Update Purger:                      30            101          0.61         0.03
Update Tactical:                   530            101         10.75         0.53
Scan Descriptor:                  2554            101         51.82         2.55
Process Input:                    1383            101         28.06         1.38
Process Output:                    450            101          9.13         0.45

Average IO bandwidth:            15596 MB per month
Average CPU Usage:                4.93 percent
----

Yup, definately clogs up my system =]

I think your misunderstanding on purpose :)

If your machine has used ALL available descriptors, not just the 1024 connections that your MUD can use, then the system would be clogged, since none of the other system processes would be able to open new descriptors.

And I don't see any reason why another process couldn't steal the descriptor that you free before you open a new one. Even if closing fpReserve is followed by opening a new descriptor, another process could easily have been giving CPU time and have used the new free descriptor, leaving you with nothing. Just because two lines comes just after eachother in a program, doesn't mean that no other running processes will be able to do anything while those two lines execute.

There, that should be clear enough.

Brian

>I think your misunderstanding on purpose :)

*grin*

>There, that should be clear enough.

Not entirely, my mud would be the last thing I'd worry about if my server would run out of descriptors, which is impossible with the linux kernel it's using. If the server runs out of memory my mud will crash, too bad =]

Also the assigned amount of file descriptors is not determined per user like you said, but per process. Though I'm sure you can talk yourself out of that one.

>There, that should be clear enough.

I understand your reasoning, just that fdReserve wasn't added to deal with the system max, but with the process max, hence it works, which I proved I think. So saying it doesn't work would be wrong. It doesn't help much either when the cleaner pulls the plug as well, perhaps we can agree on that? =]

Also the assigned amount of file descriptors is not determined per user like you said, but per process. Though I'm sure you can talk yourself out of that one.

I'm sure you are right on that one :)


I understand your reasoning, just that fdReserve wasn't added to deal with the system max, but with the process max, hence it works, which I proved I think.


How would fdReserve help with the process max? You are basicly just using one more descriptor. Having fdReserve handy will make sure (I still don't believe that will always work) that you can always fetch a free descriptor from your back pocket, but what happens at the end of that function, when you are "allocation" fdReserve again? Unless the descriptor that you used in that function was closed, you no longer have any free descriptors for fdReserve, and not all functions allocate descriptors on a temporary basis (getting a new connection for instance).

And if you where in a situation, where your MUD had used all descriptors, and you succeeded in using fdReserve, then simply by not using fdReserve to begin with, you would have one additional free descriptor, which the MUD could have used.

fdReserve still seems like a waste, it doesn't help with the process limit (it actually decreases it), and it doesn't guarantee anything with regard to the server limit, it's more of a cross your fingers and hope thing.

Brian

Not entirely sure what you mean, you close the file pointer which points to /dev/null and re-open it right away. When done you close it and point it back to /dev/null. As you can see in a non threading program there's no way something else can snatch away the file descriptor.

My test also shows that the connection is simply closed when no file descriptors are available, so that isn't an issue either.

You need the spare file descriptor to save pfiles and stuff. Which is why it was added to begin with. I can understand someone finds it the wrong way to go about things, but it works :)

Do not limit yourself to thinking your program is the only one running on the server. Other processes can snatch it away.

But I'm just repeating myself, please re-read my other posts

Brian

You don't list the kernel memory used by those socket file descriptors.
What is your connection backlog set at (i.e. listen)?

Process memory is about 100Kb per descriptor, 60K output buffer, 2K input buffer, 32K for mccp, and some other stuff. Clueless about other related memory usage that comes along with descriptors at a deeper level :)

backlog is set to 20, initializing 10 descs per second.

I'd say only rom and rom derivitives use this approach, most muds treat memory as memory, and have 1 hash table with lists of free memory blocks.

Nowadays it's faster to free and allocate directly, as pointed out by Drey, and I'd guess most people using this approach either don't know how it works and leave it be, or do not fix what's not broken (till the entire thing breaks down), or like having some memory debugging (not that merc has any of that) since the average codebase is under non stop development.

What I picked up is that the 'do it yourself memory management' was faster, because the stock merc memory manager uses more memory than direct free/alloc (can waste up to 24% of memory in some implementations WCS). Or perhaps it was a myth all along, because merc doesn't come along with a build in benchmark, or another option, gcc's/linux's memory management has been improved over the years.

I have to confess, I'm not positive which way is faster and artificial benchmarks to prove one or the other aren't going to make me feel any better. I do know my Envy derivative was noticeably (but not measured) faster when I ripped out *all* of the managed memory; however, I ripped out string management at the same time as all other memory management. Some runs through gprof had me leaning towards SSM being very slow and I just decided to nuke it all as long as I was in there. It also makes running the thing through valgrind easier -- there are many places in Envy where new structures are allocated from managed memory and not properly initialized; valgrind seems to find them better when you're letting the OS manage the memory with malloc (etc) and free calls.

The point of fdReserve is that, if you run out of file descriptors, you can open the bug report file (with the space reserved by fdReserve) and say "ran out of descriptors!".

If I'd remove all my debugging checks I think I'd get quite close to free/calloc, with ofcourse the SSM's string dupes which are faster, and used a lot in my main cpu drain: loading pfiles. Then again, I nullify all memory before re-using it, and calloc seems to do that more efficiently than memset.

The main issue with old SSM's, and generally every old program, is that they don't sacrifice enough memory for speed, my merc uses about 25% more memory for a 1000% speed gain.