[PHP] Threading Class

[SplitIce]

Ok most people would think threading in php is dificult or not possible so I whipped up this class for one of my projects in about 10 mins. Enjoy.

<?php
class Thread {
    private $pid = false;
    private $file = false;
    private $kill = true;
    
    /*
    Usage (callback function): $thread1 = new Thread('function','argument1','argument2');
    Usage (static calling class): $thread1 = new Thread(array('class name','function'),'argument1','argument2');
    Usage (object calling): $thread1 = new Thread(array($object,'function'),'argument1','argument2');
    */
    function Thread(){
        $this->file = '/tmp/'.md5(rand().rand().rand()+rand()).md5(rand().rand().rand()+rand()).'.thread';
        $pid = pcntl_fork();
        if ($pid == -1) {
            die('could not fork');
        } else if ($pid) {
            $this->pid = $pid;
        } else {
            $ar = func_get_args();
            $function = $ar[0];
            unset($ar[0]);
            $return = call_user_func_array($function,$ar);
            file_put_contents($this->file,serialize($return));
            if($this->kill) posix_kill(getmypid(),9);
            else die();
        }
    }
    
    /*
    Because php wasnt made with multi-threading in mind as soon as a thread closes it does a clean up.
    Setting this to true prevents php doing a clean up, ensure you unset any global vars you may have defined
    in the CHILD thread.
    
    Ensure this is true (default) if you experience the MySQL "Lost Connection during query" problem
    */
    function set_kill($setting){
        $this->kill = (bool)$setting;
    }
    
    /*
    Get the function return result. Returns a object.
    $return->result is where you will find any result data
    $return->error will be set if an error has occured with the error string
    */
    function result($clear=true){
        $new = new stdClass();
        pcntl_waitpid($this->pid,$status);
        if(!file_exists($this->file)){
            $new->error = 'function triggered error';
            $new->result = null;
            return $new;
        }
        $return = unserialize(file_get_contents($this->file));
        $new->result = $return;
        if($clear) unlink($this->file);
        return $new;
    }
}

//Demo
function sleep1($n){
    sleep($n);
    return $n;
}
$thread1 = new Thread('sleep1',6);
$thread2 = new Thread('sleep1',3);
echo 'Now we wait for the threads to complete their "work": ';
echo $thread1->result()->result.'-'.$thread2->result()->result."\r\n";
?>

Dont use this when loading a page with apache2, it can be done but its not recomended.
Only works on linux at this stage, requires the pcntl extension which is linux only!

SplitIce Reviewed by SplitIce on 27th Sep 2009. [PHP] Threading Class Ok most people would think threading in php is dificult or not possible so I whipped up this class for one of my projects in about 10 mins. Enjoy. <?php class Thread { private $pid = false; private $file = false; private $kill = true; /* Usage (callback function): $thread1 = new Thread('function','argument1','argument2'); Rating: 5

[litewarez]

i think the concept of this is VERY advanced for users here lol, i thought i was bad, but yea, you use this for multiple task such as down/up streaming simultaneously. i never really worked on projects that needs to use such a method but thanks for sharing.. Good code

[Hyperz]

Didn't know there was such a thing for PHP. A quick look at the sample code tells me that this is far from efficient though. Not to mention the overhead this will create since you don't control the thread but you control "something" that controls the thread, and all of this being done interpreted. Nevertheless, good to know.

[SplitIce]

The overhead that occurs due to "forking" the process in php is minimal, barely more than introducing another thread in a non interpreted language.

Its basicly starting another php executor in a child thread, howeaver all memory is referenced (using the php refcount) to the origional thread, this means that data is not duplicated for the second thread as it generally is in most mutliple thread applications (usually the programmer does this to keep things simple). Howeaver if you alter a variable then, it does create it in a new thread.

Personally when doing anything on the backend this is the best way to do things, whether its a php multiposter (Yeah Hyperz i saw your post) or a ddl submitter this is the fastest way to do things that take a long time, expecially when there is auser waiting for the results.

Oh and note that unless you are running a fcgi server (mod_fcgid, mod_fastcgi, lighttpd) you really shouldnt do this on the web server, apache does not like it (tho it is possible)

[Hyperz]

Well no I have no doubt that this can speed up some tasks a lot. What I was trying to say that this isn't something that should be used a lot on your average (shared?) hosting. Threading in PHP just doesn't seem right (it's not a strongly typed language). How does thread synchronization or locking vars work in this? I'm sure there is a good reason behind PHP not supporting threading out of the box.

[SplitIce]

Vaiable locking does not occur nor really need to occur as there is no communication (through natural methods) between threads. All variables are re allocated for the child thread if modified.

If the threads wanted to comunicate (parent-child or child-child) the best way would be an IPC pipe or a dump file (like my serialized return result).

The reasoning behind php not supporting threading by default as it isnt neccearry for alot of things, its an extension like mysql. And just like mysql it can be used when not needed etc.

[Dman]

How do you make sure both threads complete their work? In java we join those.

[SplitIce]

What do you mean, as soon as you request the result it waits till the thread is finnished, I suppose I could add a check function, I didnt need that in my application.

[SplitIce]

While im at it here is a PoC for a script, its a downloader, downloads all the arcade gamess from igames.origon.dk, the server is shit slow so we do 20 connections at a time and we can increase the total script speed by about 500%

<?
set_time_limit(0);

define('IN_PHPBB', true);
$phpbb_root_path = (defined('PHPBB_ROOT_PATH')) ? PHPBB_ROOT_PATH : './';
$phpEx = substr(strrchr(__FILE__, '.'), 1);
include($phpbb_root_path . 'common.' . $phpEx);
ini_set('memory_limit','1000M');
function unset_error_handler()
    {
        while (set_error_handler(create_function('$errno,$errstr', 'return false;'))) {
            // Unset the error handler we just set.
            restore_error_handler();
            // Unset the previous error handler.
            restore_error_handler();
        }
        // Restore the built-in error handler.
        restore_error_handler();
    }
    
restore_exception_handler();
unset_error_handler();


function login(){
    global $ch;
    $ch = curl_init('http://igames.origon*****forum/ucp.php?mode=login');
    curl_setopt($ch,CURLOPT_POST,true);
    $post = array(
        'username'=>'splitice',
        'password'=>'-',
        'autologin'=>'checked',
        'login'=>'Login'
    );
    curl_setopt($ch, CURLOPT_COOKIEJAR, "/tmp/arcade.txt");
    curl_setopt($ch, CURLOPT_COOKIEFILE, "/tmp/arcade.txt");
    curl_setopt($ch,CURLOPT_POSTFIELDS,$post);
    curl_setopt($ch,CURLOPT_RETURNTRANSFER,true);
    return curl_exec($ch);
}

function readHeader($ch, $header) {
        //extracting example data: filename from header field Content-Disposition
        if(stripos($header,'Content-Type: application/zip; name="')===0){
            $fname = substr($header,strlen('Content-Type: application/zip; name="'),-3);
            global $__fname;
            $__fname = $fname;
        }
        return strlen($header);
    }

function extractCustomHeader($start,$end,$header) {
        $pattern = '/'. $start .'(.*?)'. $end .'/';
        if (preg_match($pattern, $header, $result)) {
            return $result[1];
        } else {
            return false;
        }
    }


function fetch_page($url){
    global $ch;
    curl_setopt($ch,CURLOPT_POST,false);
    curl_setopt($ch,CURLOPT_URL,$url);
    curl_setopt($ch, CURLOPT_HEADERFUNCTION, 'readHeader');

    $d=  curl_exec($ch);
    return $d;
}

// Start session management
$user->session_begin();
$auth->acl($user->data);
$user->setup();

include($phpbb_root_path . 'includes/arcade/arcade_common.' . $phpEx);
include($phpbb_root_path . 'includes/arcade/arcade_admin_class.' . $phpEx);
include($phpbb_root_path . 'includes/functions_compress.' . $phpEx);
        // Initialize arcade auth
$auth_arcade->acl($user->data);
        // Initialize arcade class
        $arcade = new arcade_admin();

$download_data = $arcade->get_remote_data("http://igames.origon*****forum/arcade.php?mode=download&type=data");
foreach($download_data["categories"] as $cat){
    $games = array();
    $download_list = $arcade->get_remote_data("http://igames.origon*****forum/arcade.php?mode=download&type=list&c=".$cat['cat_id']."&start=0&sk=asc&sd=$sort_dir&per_page=50");
    $games = $download_list['games'];
    for($i=1;$i<=ceil($download_list["total"]/50);$i++){
        echo 'Downloading page: ',$i,"\r\n"; 
        $download_list = $arcade->get_remote_data("http://igames.origon*****forum/arcade.php?mode=download&type=list&c=".$cat['cat_id']."&start=".($i*50)."&sk=asc&sd=$sort_dir&per_page=50");
        $games = array_merge($games,$download_list['games']);
    }
    foreach($games as $k=>&$g){
        $res = $db->sql_query('SELECT * FROM phpbb_arcade_games WHERE game_name_clean="'.$db->sql_escape($g['game_name_clean']).'"');
        if($db->sql_fetchrow($res)) {
            unset($games[$k]);
            echo 'Skipped ',$g['game_name_clean'],"\r\n";   
        }else {
            echo 'Downloaded ',$g['game_name_clean'],"\r\n"; 
            login();
            $f = '/tmp/'.md5(rand().rand()).'.zip';
            usleep(100);
            $g['file_data'] = new Thread('fetch_page','http://igames.origon*****forum/arcade.php?mode=download&type=acp&g=' . $g['game_id'].'&use_method=.zip'); 
        }
    }
    foreach($games as $g){
        $d = $g['file_data']->result()->result;
        file_put_contents($f,$d);
        $g['file_data'] = $f;
        $g['filename'] = $__fname; 
    }
    foreach($games as $v){
        usleep(200);
        try {
        $compress = new compress_zip('r', $v['file_data']);//$phpbb_root_path . $arcade->config['unpack_game_path'] . $game
        $tmp_path = 'store/' . md5(unique_id()) . '/';
        $compress->extract($phpbb_root_path . $tmp_path);
        $compress->close();
        
        $is_ibpro = $is_xml = $is_old = $game_type = false;
        $game_name = $file_functions->remove_extension($v['filename']);
        if (substr($game_name, 0, 5) == 'game_'){
                $is_ibpro = true;
                $game_name = str_replace('game_', '', $game_name);
        }

        @unlink($phpbb_root_path . $arcade->config['unpack_game_path'] . $v);
        
        if(file_exists($phpbb_root_path . $tmp_path . 'gamedata')) $file_functions->copy_dir($phpbb_root_path . $tmp_path . 'gamedata', $phpbb_root_path . 'arcade/gamedata');
        
        $file_functions->copy_dir($phpbb_root_path . $tmp_path, $phpbb_root_path . $arcade->config['game_path'] . $game_name);   
        $file_functions->delete_dir($phpbb_root_path . $tmp_path); 
        echo 'Extracted ',$v['game_name_clean'],"\r\n"; 
        } catch(Exception $e){
            echo $e->getMessage(),"\r\n";
        } 
        
    }
    die('end');
}

class Thread {
    private $pid = false;
    private $file = false;
    private $kill = true;
    
    /*
    Usage (callback function): $thread1 = new Thread('function','argument1','argument2');
    Usage (static calling class): $thread1 = new Thread(array('class name','function'),'argument1','argument2');
    Usage (object calling): $thread1 = new Thread(array($object,'function'),'argument1','argument2');
    */
    function Thread(){
        $this->file = '/tmp/'.md5(rand().rand().rand()+rand()).md5(rand().rand().rand()+rand()).'.thread';
        $pid = pcntl_fork();
        if ($pid == -1) {
            die('could not fork');
        } else if ($pid) {
            $this->pid = $pid;
        } else {
            $ar = func_get_args();
            $function = $ar[0];
            unset($ar[0]);
            $return = call_user_func_array($function,$ar);
            file_put_contents($this->file,serialize($return));
            if($this->kill) posix_kill(getmypid(),9);
            else die();
        }
    }
    
    /*
    Because php wasnt made with multi-threading in mind as soon as a thread closes it does a clean up.
    Setting this to true prevents php doing a clean up, ensure you unset any global vars you may have defined
    in the CHILD thread.
    
    Ensure this is true (default) if you experience the MySQL "Lost Connection during query" problem
    */
    function set_kill($setting){
        $this->kill = (bool)$setting;
    }
    
    /*
    Get the function return result. Returns a object.
    $return->result is where you will find any result data
    $return->error will be set if an error has occured with the error string
    */
    function result($clear=true){
        $new = new stdClass();
        pcntl_waitpid($this->pid,$status);
        if(!file_exists($this->file)){
            $new->error = 'function triggered error';
            $new->result = null;
            return $new;
        }
        $return = unserialize(file_get_contents($this->file));
        $new->result = $return;
        if($clear) unlink($this->file);
        return $new;
    }
}
?>