PHP5 Opcode caching and memory storage with APC/XCache in command line interface (CLI) or cron

Opcode accelerators (such as Opcode, APC, Xcache) can't work on CLI scripts. I mean: can't work for your good.

Why access accelerator/cache in CLI mode?

Many applications work thanks to APC/Xcache, including saving user keys into memory for fast access. But if you need to run a cron script, for example to update users' statuses, or to clear the cache, you're stuck:
- you don't have opcode caching (okay, you're running in a cron, so you don't care about speed)
- you can't access user keys, so if it's part of your application (for example timestamp key to clear all caches on a specific object), you can't use it at all.

What are opcode accelerators?

Just a little explanation for those who are not familiar with opcode accelerators. PHP scripts are compiled on the fly: PHP script is opened, compiled into bytecode (or opcode), then the bytecode is executed. Each time you run the script, PHP will compile it, exactly the same way, to execute it.
That's where the accelerator takes place: it keeps the bytecode in a cache (in RAM), so that the next time you run the script, PHP has only to execute the bytecode.

But in CLI, as PHP is run outside of a service, the cache is lost at the end of the execution. So enabling APC's cache (apc.enable_cli = 1), is a bad idea.

The idea: access your PHP server directly

One solution is to access PHP through your webserver, by making an HTTP call. But if you need to run a script in CLI, it may last one or two hours: your webserver will shut down the connection before that. And you'll have to set dramatically high timeouts for everyone accessing your webserver...

The solution is to access PHP the same way your webserver does:
- if you're using mod_php: I'm sorry for you, there's nothing interesting here for you.
- if you're using fastcgi: talk to your fastcgi server!
- if you're using php-fpm: talk to it!

I can't say that this will work for you, but it worked for me with lighttpd + fastcgi. I'm pretty sure it will work with php-fpm, and with any nginx setup. Same thing for Apache, as long as it's using fastcgi/php-fpm.

The solution

FastCGI client

The language here is FastCGI. So you need a client to speak this language. I chose this PHP FastCGI Client.

I must state that I've changed the request method of PHP FastCGI Client, so that it dumps what it reads:
public function request(array $params, $stdin, $handleContent = null)
{
$id = $this->async_request($params, $stdin);
return $this->wait_for_response($id, 0, $handleContent);
}

and
public function wait_for_response($requestId, $timeoutMs = 0, $handleContent = null)

and
$this->_requests[$resp['requestId']]['response'] .= $resp['content'];
if ($handleContent)
{
$handleContent($resp['content']);
}


Main script

This script is NOT accelerated. It doesn't have any access to shared APC. So that's the only «slow» script.

I'm using lighttpd, so I can find something like:
fastcgi.server = ( ".php" =>
(( "socket" => "/tmp/php-fastcgi.socket",
"bin-path" => "/usr/local/bin/php"
))
)

Hence, the fastcgi server is accessible through it's first socket: /tmp/php-fastcgi.socket-0

So here is the fastcgi.php:
<?php
use Adoy\FastCGI\Client;
ob_implicit_flush(); // dump text as soon as it comes in
set_time_limit(0); // runs with no time limit
include 'Client.php';
$client = new Client('unix:///tmp/php-fastcgi.socket-0', 0); // on php-fpm, you should use new Client('localhost', 9000)
/* Timeout 15 min */
$client->setReadWriteTimeout(900000);
echo "Starting FastCGI call\n";
$arguments = array('script' => $_GET(0)); // this is based on php-cgi call. If you use php/php-cli, you may use $_SERVER['argv']
$request = [
'REQUEST_METHOD' => 'GET',
'SCRIPT_FILENAME' => '/full/path/to/cgi.php',
'QUERY_STRING' => http_build_query($arguments)
];
try
{
$response = $client->request($request, null, create_function('$response', 'echo $response;'));
}
catch (Exception $e)
{
echo $e->getMessage();
}
echo "Ending FastCGI call\n";


Child script

The main script will launch a call to:
'SCRIPT_FILENAME' => '/full/path/to/cgi.php',


I've chose to create a wrapper, so that main parameters are already set for all of the scripts I'll call through the fastcgi.php.

So here is the cgi.php:
<?php
set_time_limit(0); // no time limit
ob_implicit_flush(); // dump anything as soon as possible
$buffering = (int) ini_get('output_buffering'); // buffering length, most of the time: 4096 bytes
/**
* Dump the $string so that it fits in the buffering length and gets flushed automatically
*/
function fastEcho($string)
{
global $buffering;
echo str_repeat("\033(0m" // /!\/!\ bad editor formatting, please replace ( with an opening bracket
, max(1, ceil(($buffering - strlen($string) % $buffering) / 4))).$string;
flush();
}
// call the real PHP script
// use $_GET for variables, to find out what is the script set to fastcgi.php
include str_replace(array('.', '/'), array('', ''), $_GET['script']).'.php';


Calling the script

Some parts should be missing (using php-cli instead of my example with php-cgi), and you'll have to find out what $_GET variables in cgi.php you should use to call the final script, but the main call would be something like:
php-cgi /path/to/fastcgi.php myscript parameter1 parameter2


The function «fastEcho()» is useful to echo() text and show it immediately, as the output buffers until it is 4096 bytes long.

I should set a gist for that, so ask for it if you want one!

Photo credit: Random Access Memories, album by Daft Punk.