Scaling ModPerl
ModPerl provides an important performance enhancement to DataBreeze, yet there are issues with the current implementation of ModPerl that should be understood and addressed. The main issue centers around memory consumption of the ModPerl server and how best to optimize the platform. The ModPerl guide is the best source for ModPerl documentation and specifically address ModPerl Performance Strategy. The suggested platform setup for DataBreeze is found in the Adding a Proxy Server in http Accelerator Mode
ModPerl has a large memory footprint
Mod_perl server processes are generally quite large compared with standard server processes. The typical static build (Apache child) server consumes around 1.x MB and does not grow. DataBreeze mod_perl child processes may start at 10 MB and slowly grow in size, consuming all memory if left unchecked. This problem can be magnified if executed code contains any leaks.
The problem lies in slow client connections
- Client connects from an ISP with a typical saturated, oversubscribed internet connection
- Client sends a request to the mod_perl server
- Mod_perl process executes the request in less than a second
- The mod_perl server process must wait until the client has finished before disconnecting
- The client is slow, the mod_perl process dribbles the page back to the client for 20 seconds
- Meanwhile, the server has stacked up 20 new connections
Result: The expensive mod_perl process becomes tied up serving a single client while others wait
Solution: buffer the mod_perl server by using a front-end proxy
A Front-end server has mod_proxy module which passes script requests back to ModPerl server.
- Client connects to lightweight front-end
- Front-end ProxyPasses request to fat mod_perl server
- Mod_perl server executes and returns document to front-end
- Lightweight Front-end is responsible for returning (now static) document to client
- Meanwhile, Mod_perl is finished and can serve another request
Result: Lightweight Front-end handles 'dribbly' client requests and heavyweight mod_perl is used only as needed
Using mod_rewrite to separate calls for static images and HTML documents
Serving static documents from the heavyweight mod_perl server is a waste, so set up mod_rewrite in the lightweight front-end machine to keep staic HTML and image requests local while ProxyPassing everything else.
For more information on this solution, look at the mod_perl guide.
