March 28, 2024, 07:06:26 PM

News : LinuxSolved.com Linux Help Community Forum..


Author Topic: Memory optimization for child processes  (Read 4201 times)

Offline umaramnath

  • New Member
  • Posts: 1
Memory optimization for child processes
« on: September 06, 2010, 11:59:34 AM »
Hello everyone,

I have been following the forum for a while now. This is my first post here.

I work on Linux for ARM processor for cable modem. There is a tool that I have written (as the job demands) that sends/storms customized UDP packets using raw sockets. I form the packet from scratch so that we have the flexibility to play with different options. This tool is mainly for stress testing routers.

The details are here.

I actually have multiple interfaces created. Each interface will obtain IP addresses using DHCP. This is done in order to make the modem behave as virtual customer premises equipment (vcpe).

When the system comes up, I start those processes that are asked to. Every process that I start will continuously send packets. So process 0 will send packets using interface 0 and so on. Each of these processes that send packets would allow configuration (change in UDP parameters and other options at run time). Thats the reason I decide to have separate processes.

I start these processes using fork and excec from the provisioning processes of the modem.

The problem now is that each process takes up a lot of memory. Starting just 3 such processes, causes the system to crash and reboot.

I have tried the following:-
1-I have always assumed that pushing more code to the Shared Libraries will help. So when I tried moving many functions into shared library and keeping minimum code in the processes, it made no difference to my surprise.

2-I also removed all arrays and made them use the heap. However it made no difference. This maybe because the processes runs continuously and it makes no difference if it is stack or heap?

3-I suspect the process from I where I call the fork is huge and that is the reason for the processes that I make result being huge. I am not sure how else I could go about. say process A is huge -> I start process B by forking and excec. B inherits A's memory area. So now I do this -> A starts C which inturn starts B will also not help as C still inherits A?. I used vfork as an alternative which did not help either. I do wonder why.

I would appreciate if someone give me tips to help me reduce the memory used by each independent child processes.

Kindly do let me know if you need more details or clarification.

Thank you.

Offline dragoncity99

  • LST CareTaker
  • Experienced
  • *****
  • Posts: 551
Re: Memory optimization for child processes
« Reply #1 on: October 02, 2010, 03:18:43 AM »
1. Watch out for all possible resources trending.

2. If  you it's true your network eats up memory, try play around with the udp send/recv buffer in /etc/sysctl.conf, and play with the minimum memory reserve for your OS itself. You starve linux memory itself. It's gonna get OOM to cause hang or processes killed.

3. Lastly, hope it's not an extreme number the tool generates, if it can be overrided, i say start with something small first and then step up the value gradually. You need it anyway for reporting or graphing rite ? :)