PHP
Raiting:
14

CThread - multithreading for PHP with blackjack


In the Web you can find a lot of solutions to emulate multithreading in php. Most often they are based on the forks, but there are variations about using curl, proc_open and etc.

I did not like the alternatives that I found, so I had to write my own solution.
Here is the following set of requirements:

• Use of the forks
• In-sync state with the interface in the absence of the necessary extensions
• Multiple use of child processes
• A full data exchange between processes. That is, running with the arguments and getting results at the end
• The event exchange between the child process-"thread" and the basic process at work
• Handling the thread pool with the multiple use, transferring arguments and getting results
• Error handling
• Timeouts for the work performance, waiting for work thread, initialization
• Maximum performance

The result is a CThread library.

Here is a link to the sources:
github.com/amal/CThread

Description


CThread provides a simple interface to create thread classes. They actually use separate processes to operate asynchronously, but you should not worry about it. You can send events from the thread, return results, use one thread many times, transferring the run arguments or create a pool of 16 threads and not paying any attention to the fact that work is done in different processes.

Alternatively, you can easily test the performance of libraries in different modes, selecting the optimal number of threads and the option of data between processes, specifically for your configuration.

In order to work properly the following extensions are needed: libevent, posix and pcntl.

The library uses LibEvent and paired sockets for communication between the processes. It supports 5 options for data transfer (arguments, results and data events)!

Here are options with the data performance. It was tested with a pool of eight threads for Intel Core i7 2600K 3.40 Ghz (Ubuntu 11.04 on virtual VMware). Here are given the average results for 10 reruns of the test in jps (jobs per second).
jps Description
1 6501 Data transfer in serialized form through the same sockets. The default option.
2 6625 The same, but with igbinary serialization (the best performance option.) It is used as the default if igbinary installed.
3 6194 System V Memory queue (sysvmsg)
4 6008 System V Shared memory (sysvshm)
5 6052 Shared memory (shmop)
Extensions are automatically selected to work with the sockets. If it is available, then the extension of sockets is used, which gives better performance. Otherwise, the stream is used.

All available signals obey in the child process. By default, all of them (except SIGWINCH and SIGINFO) should end work. But this can easily be overridden by creating a method with a signal name in a thread class, for example: sigWinch.

Also, all signals are intercepted by default in the parent process. This can be changed by setting a class parameter listenMasterSignals in false. In this case, SIGCHLD will only be processed. Also, there can be easily added additional handlers by creating a static method with the name m <name of the signal>. For example: mSigTerm.

If a child process fails for whatever reason, the class automatically forks when running a new task, so you do not have even to think about it.

The child process checks the existence of a parent one from time to time. If it suddenly fails then the child process will automatically be terminated.

All resources that are being used by the thread or pool of threads will be automatically cleared when a destructor will be called. But they can be forced to clean up if the cleanup method will be called. In this case, thread/pool no longer could be used.

The thread is initialized in advance in the standard settings, right away when a class is created. If a parameter prefork is set in false, then the fork will only occur when running a task.

In general, there are a lot of configurable parameters. When the child process name is changed after fork (parameter pName of constructor), there are the timeout for the duration of the task (timeoutWork), timeout for the maximum waiting time of tasks by the child process (timeoutMaxWait), timeout for the time pre-initialization (timeoutInit), buffer sizes for reading of sockets (pipeReadSize , pipeMasterReadSize).
It is possible to disable the multitasking mode for threads (multitask). In this case, each time after a task completion the child process will fail and fork again for the next run. It will noticeably reduce the performance.

Tested code is documented in details, and the examples of use can be found and run in the file example.php.
More complex examples with error handling can be found in code of unit test.

There is a debug mode that displays very detailed information, what is happening and where.

Examples of use:


The main feature is the maximum simplicity. If you just want to run something in a separate "thread" use the following code:

class ExampleThread extends CThread
{
protected function process()
{
// Some work here
}
}

$thread = new ExampleThread();
$thread->run();

If there is everything for the proper work, the task will be executed asynchronously. If not, then everything will still work, but in synchronous mode.

Send argument and receive result of processing:

class ExampleThread extends CThread
{
protected function process()
{
return $this->getParam(0);
}
}

$thread = new ExampleThread();
$result = $thread->run(123)->wait()->getResult();
Triggering events from thread:

class ExampleThread extends CThread
{
const EV_PROCESS = 'process';

protected function process()
{
$events = $this->getParam(0);
for ($i = 0; $i < $events; $i++) {
$event_data = $i;
$this->trigger(self::EV_PROCESS, $event_data);
}
}
}

// Additional argument.
$additionalArgument = 123;

$thread->bind(ExampleThread::EV_PROCESS, function($event_name, $event_data, $additional_arg) {
// Event handling
}, $additionalArgument);

$events = 10; // number of events to trigger

$thread = new ExampleThread();
$thread->run($events)->wait();
Finally, use pool with 8 threads:

$pool = new CThreadPool('ExampleThread', $threads);

$num = 25; // Number of tasks
$left = $num; // Remaining number of tasks

do {
// If the pool has waiting threads
// And we still have tasks to perform
while ($pool->hasWaiting() && $left > 0) {
// You get thread id after start
$threadId = $pool->run();
$left--;
}
if ($results = $pool->wait($failed)) {
foreach ($results as $threadId => $result) {
// Successfully completed task
// Result can be identified
// with thread id ($threadId)
$num--;
}
}
if ($failed) {
// Error handling.
// The work is completed unsuccessfully
// if the child process has died at run time or
// work timeout exceeded.
foreach ($failed as $threadId) {
$left++;
}
}
} while ($num > 0);

// Terminating all child processes. Cleanup of resources used by the pool.
$pool->cleanup();

Here are the results of performance testing


Tests were run on two machines with Ubuntu 11.04.
First - Intel Core i3 540 3.07 Ghz
Second - Intel Core i7 2600K 3.40 Ghz (Ubuntu is on the virtual VMware).

Here are results to estimate an increase in performance.
These are the average results for a series of 10 reruns of the test in jps (jobs per second).

The threads execute the following task:

for ($i = 0; $i < 1000; $i++) {
$r = mt_rand(0, PHP_INT_MAX) * mt_rand(0, PHP_INT_MAX);
}

The first result is indicated for synchronous mode (without forks).
I have not tried 18 and 20 threads on the first configuration, as for 12 has already begun drop in the performance. <td 2188</td>
The number of threads The first configuration The second
0 553 763
1 330 669
2 580 1254
4 1015
8 1040 2618
10 1027 2719
12 970 2739
16 958 2904
18 - 2830
20 - 2730
image

As you see the performance goes up at least in 2-3 times!

The code that performs a series of tests with the needed parameters is in a file test.php. So you can easily test the performance and select the optimal number of threads for yourself.

Once again here is a reference to the sources:
github.com/amal/CThread

I would be very glad if the library will be useful for anyone.

Thank you for your attention!
xially 7 february 2012, 13:10
Vote for this post
Bring it to the Main Page
 

Comments

Leave a Reply

B
I
U
S
Help
Avaible tags
  • <b>...</b>highlighting important text on the page in bold
  • <i>..</i>highlighting important text on the page in italic
  • <u>...</u>allocated with tag <u> text shownas underlined
  • <s>...</s>allocated with tag <s> text shown as strikethrough
  • <sup>...</sup>, <sub>...</sub>text in the tag <sup> appears as a superscript, <sub> - subscript
  • <blockquote>...</blockquote>For  highlight citation, use the tag <blockquote>
  • <code lang="lang">...</code>highlighting the program code (supported by bash, cpp, cs, css, xml, html, java, javascript, lisp, lua, php, perl, python, ruby, sql, scala, text)
  • <a href="http://...">...</a>link, specify the desired Internet address in the href attribute
  • <img src="http://..." alt="text" />specify the full path of image in the src attribute