Motivation

I once was a veteran Linux user and knew the benefit of Logical Volume manager (LVM). To be simple, it can make several physical disks merged into one drive and you can manage the adding, removing and replacing of physical disk dynamically. Time machine require 2-3 times larger than your working harddisk. Even though multiple disks can be used in Time machine backup, because Time machine store the initial image on individual disks, if your working disk image too large, there’s not much room for “addition modification images”. Fusion drive, is a tiered harddisks/software combination solution in Mountain Lion. Apple released it in 2012 iMac and Mac mini. A hack make it available on older Mac models.. The disk reading/writing boost does attract me, but as a customer whose iMac is still under AppleCare protection, I believe in that most of Mac users in my condition won’t take the risk to open their mystery Mac box and do the trick to add an internal SSD. To summarize, I need a solution:

• an external tiered storage system, which can combine my mobile disk and reuse them.
• If it’s possible, an alternative solution to Fusion drive is good too.
• As little tricks as possible. I know a lot of workarounds may work, but all of them may introduce new bugs which need new workarounds.

Benchmark

Firstly I would like to calculate what maximum rate I can get from each interface. According to harddsk performance report of iMac 2011, my model speed is around 160~170 (MB/s). I pick up review from Tom’s hardware, seems performace vary in a quite large gap.

But I use Blackmagic Disk Speed Test to test my own 7200rpm HDD WDC WD1001FALS-403AA0 in iMac 2011 27 inch box.

My benchmark seems not good and not consistent with this post. How about others? Fusion drive in 2012 iMac can achieve Write: 249 and Read: 382 (MB/s).

Thunderbolt or not?

Thunderbolt (TB) is still a synonym to expensive. If I buy USB 3.0 external disk, then I need a TB to USB 3.0 dock/hub. According to wiki, there are 3 available docks right now.

• Belkin Thunderbolt Express Dock
  • Price: 400USD without TB cable
• Matrox DS1
  • Price: 249USD without TB cable
• Sunix Thunderbolt Dock
  • Price: not available

Other manufacturers not listed in wikipedia:

• Sonnet also offer thunderbolt adpater as well as storage solution, but still, extremely expensive

The price is insane. 400USD can cover the gap between 2011 iMac and 2012 iMac, and it’s certainly overpriced.

External HDD/SDD performance

• Western Digital My Book Studio 3TB WDBC3G0030HAL
  • Performance:read speed of 88.1MB/s and a write speed of 78.6MB/s
  • Price: 1390HKD
  • Interface which achieves the speed: Firewire800
• Toshiba Canvio Desk 3TB (HDWC130XK3J1)
  • Performance: read speed of 147.4 MB/s and a write speed of 136.8 MB/s.
  • Price : 146.94USD
  • Interface which achieves the speed: USB 3.0
• Western Digital My Book Essential 4TB (WDBACW0040HBK)
  • Performance: 126.1MB/s on sequential write and 127.4 MB/s on sequential read
  • Price:200USD
  • Interface which achieves the speed: USB 3.0
• Elgato Thunderbolt SSD
  • Performance: files written at 184MB/s and read at 170.7MB/s
  • [Price: 280USD for 120GB]
  • Interface which achieves the speed: Thunderbolt
• LaCie Little Big Disk
  • Performance : 70 MB/sec reads and 250 MB/sec writes
  • Price: 330USD
  • Interface which achieves the speed: Thunderbolt
• Seagate GoFlex Desk Thunderbolt Adapter (STAE122)
  • Paired with Intel SSD 520
    • Performance: 95MB/s for USB 3.0 or 291MB/s for Thunderbolt, 450MB/s for SATA
    • SSD price: 140USD for 120GB
  • Paired with 1TB FreeAgent GoFlex portable hard drive
    • Performance: 30MB/s for USB 2.0, 80MB/s for USB 3.0, 80MB/s for Thunderbolt
  • Price for GoFlex Adapter:94USD
• Akitio Neutrino Thunderbolt Edition
  • Performance: read at 464MB/s and write at 311MB/s
  • Price: 250USD 120GB SSD including TB cable
  • Interface which achieves the speed: Thunderbolt

Fusion Drive on older Mac

As I mentioned previously. It’s seems viable to make older Mac support Fusion drive, but it may be not stable.

1
2
3
4

# create Fusion drive
$ diskutil cs create bla disk1 disk7
$ diskutil cs list
$ diskutil coreStorage createVolume DE85044F-EADA-4F26-93B7-8CD0ADF006EC jhfs+ blub 500g             

The author also claimed that Fusion drive can combine external SSD/HDD or even 2 HDD together, sounds promising. Follow up guide points out one note need to pay attention:

• Do Not Use a Drive Made Up of Multiple Partitions

Alternatives Tiered storage on Mac

Disk Utility is handy and Logical Volume Group (LVG) in Mac was introduced in Lion, named CoreStorage. IBM developerWorks has a good explanation to LVG. Other than native OS X LVG, we can setup an NAS Linux host and connect to it via network. Actually, there is commercially available products.

Due to the native support, other 3rd party solutions seem not necessary. Raid 0, surely a viable solution, and there are many on the market. For instance, Lacie.

ZFS itself, is not only a filesystem but also LVG. MacZFS is the implement right now and in there site there is clear history description about ZFS on Mac. Alex Blewitt, the original MacZFS engineer, considers this implementation already stable.

Summary

It is relatively cheap to make a SSD+adapter solution, but the overall price still as high as 250USD. At this stage, I’m better staying at what I really need: a larger Time Machine backup disk. A Western Digital My Book Studio 3TB WDBC3G0030HAL seems a rather good choice. In the future, if the thunderbolt price drops to an affordable level, then surely I want to enjoy the benefit of high speed storage. Thunderbolt adapter with 2 SATA interfaces, and pre-formatted to form a single Fusion Drive combining SSD and HDD, would be very attractive.

I used to be an adventurer like you, then I took an arrow in the knee. The Elder Scrolls V: Skyrim

I played AD&D game Neverwinter Nights on my Linux box 3 years ago. At that time, other than the original campaign and 2 expansion packs, I was on the online module Neversummer server and that’s a really good experience.

Last year, I bought Neverwinter Nights 2 via Mac App Store but always didn’t have time to play it. I finally tried it last months, with about 2-3 days playing. It’s still good, but the Mac version seems buggy and it’s removed from Mac App Store now. I would like to know the current available AD&D games on Mac right now. So I did some googling.

• Neverwinter Nights 1: It’s for PowerPC, after Apple removed rossetta, it’s not playable on Lion and Mountain Lion.
• Neverwinter Nights 2: mostly playable. But I encounter some problems and stuck at the entrance of Ammon Jerro’s Haven.
• Neverwinter Nights Online: Mainland China server still not available. While cannot logon to North American Server.
• Dungeon and Dragon Online: The game is free, but you need to pay for some classes. Mac client is in Beta stage. I cannot login when I tried it. Hope it will be fixed soon.
• Baldor’s Gate Enhanced Edition: Mac version is underway. But iPad version seems not so exciting.
• Dungeons & Dragons: Heroes of Neverwinter: Every time I try to load the site, facetime showes “Not found”

Find CPU info:

$ sysctl -n machdep.cpu.brand_string

my result is:

Intel(R) Core(TM) i5-2500S CPU @ 2.70GHz

Rebuild launchpad database:

$ rm ~/Library/Application\ Support/Dock/*.db; killall Dock

Play DVD

Go to System -> CDs & DVDs -> When you insert a video DVD, set it to “ignore”. And use MplayerX or VLC to play DVD. Because DVD player in the system will override region code in firmware, the you can only change this code 5 times.

Enable terminal locate command

$ sudo /usr/libexec/locate.updatedb

Use PowerPC Apps

Rosseta was removed from Lion and no longer supported. To use applications in PowerPC executables, you need to use Parallel Desktop to install a Snow Leopard Server.

Remove duplicate entries in “Open With” menu

$ /System/Library/Frameworks/CoreServices.framework/Frameworks/LaunchServices.framework/Support/lsregister -kill -r -domain local -domain system -domain user

Useful shortcuts

• F9 : Mission Control
• F10: Single current window
• F11: Desktop
• F12: Dashboard
• Force quit: Command+Option+Shift+Esc (3 seconds)
• Go to Line End: Ctrl+E
• Go to Line Beginning: Ctrl+A

外婆姓潘，名群辉，高小毕业。在长沙黄才镇出生，长大。在那个年代，会认字的外婆隐然已是知识分子了。2天后，就是外婆88岁的生日了。之所以记得年纪，是我知道外婆刚好比我大60岁，整整一个甲子。外婆在我的童年，扮演了非常重要的角色。记得小时候，常常被长辈们问的一个问题便是：

“你最爱的人是谁啊？”

我的答案自然是妈妈。

“那第二呢”？

“外婆啊！”

小时候的我还是挺憨厚的，答案并不会随着问的人不同而不同，但这里面，其实应该还是有着小九九的：妈妈每天都在身边，而且不仅有胡萝卜，还有大棒时刻准备着，外婆虽好，但是只有假期才回老家，且没有大棒，恩威并济便少了些威势，于是便只能是第二位了。

听妈妈说，我出生的时候，父亲什么都不懂，听说产妇需要吃鸡蛋，于是便兴冲冲的提了一整篮子的熟鸡蛋上医院，让人哭笑不得。母亲那时候也是任性的小姐，如果不是外婆的匡扶指导，我这个过敏体质，现在真不知道会是什么样。

外婆不算善于讲故事，但是却做得一手好菜：萝卜皮，茄子皮，西瓜皮，冬瓜皮，红烧肉，腌鱼，腊肉……数之不尽。外婆家便是我的童年的“永无乡”——只要是回老家，我的欢呼雀跃里，便藏着饕餮的心思，也总会长胖个几斤回来。众多孩子里，我总觉得我是特别受宠的，各种无理取闹，全然不用顾忌后果。细细思量，外婆的好已经全融化在一个“好”字里了——查无可考，却又千言万语。

举家搬迁到南郡之后，和煦的气候没能温暖我孱弱的身子。日子过得也只能算是紧巴巴的，母亲父亲压力都大，我又面临升学。这个时候，又是外婆冲到了前线，照顾我。那个时候的我，表面乖巧，实际上充满了青春期的叛逆。南方的天气对外婆来说，定然是难受的，常常看到外婆买菜后，拿着自己的小毛巾，不停的擦汗。有一次班主任来家访，外婆虚报了4岁年龄：“我八十岁了”，班主任啧啧称奇。现在想来，称奇的不单指是康健，更多的是这个年龄还在为子女操劳的感动吧。

随着年纪长大，我的身体素质也稍好了些，开始知道外面的世界，开始憧憬未知的生活，相信自己的努力能够引领自己到理想的目标，同时，家居生活也便从生活的重心里移开了，回老家的次数日渐稀疏。现在每次回老家，外婆总会絮絮叨叨的说小时候的故事，说我如何会做纸工，如何叫她“猪八戒”，如何偷偷地藏东西——还有已经成为家庭笑料的那个“香蕉在楼上”的故事。也许在外婆的心里，我们这些孙子辈的，永远是那个记忆里的小不点。外婆用心血浇灌出的种子，都发芽长大了，有些甚至已经有了自己的种子，小苗。外婆在这里，那个老家就还是印象中的老家，是那个无限疲惫之后，总能安心依赖的老家。

外婆与外公，表观上看，恰似金庸小说里的胖瘦二头陀。外婆大外公两岁，贫农出身，丰满，尤为怕热，外公却清矍，且有呼吸道的毛病，畏寒。性格上也大相径庭，外婆说干就干，外公却谋定而后动。但是不管怎样，这对冰火搭配的夫妻，感情定是极深的。外公在世的时候，会在后园里种些蔬果瓜菜，我印象里的冬瓜，足有近一米长，做腌菜用的白菜，也是极大极水嫩的，长这么大，只有外公那里，才有这么好的菜。可惜外公去世后，体力所限，逐渐的无人拾掇，便也荒废了。每年回去，在二楼客厅，拜祭外公的时候，相框里外公的慈容笑貌一丝不变，旁边絮叨着“老公子，外孙子来看你了”的外婆，却渐渐的老了。

外婆勤快，麻利，每日里早早起来，手脚不停要忙到晚上大家吃完饭，洗漱完；身体虽有隐疾，却有一双有力的大手和在这个年纪仍然强健的腿脚——楼梯上下二楼，来来回回，并不需要挽扶，每每我试着，外婆总是说，“莫扶，我自己扶着扶手能走”；做商业公司的会计，子女里也或多或少，全部和财会相关，这大概就是润物细无声的道理吧；跟了外公后，众多子女，又抚养了姑奶奶还有过世的舅公的两个孩子，家务操劳。印象里，买菜都是外公的工作，外婆几乎不需要出家门。外婆虽然少出外走动，但是心里的帐是门儿清的，迎来送往，礼数不能缺。外婆也爱美，不管身体如何，始终也是要拾掇的贴贴切切的，头发要梳，衣服要整齐，姿态要好：那个年代的美，不怎么需要太多的化妆品，保养，但是却并不少了“精气神”，用外婆的话说，“亲亲切切的”，格外有范。

长年在外，数年才回去一次，但是外婆身体力行的教诲，早已不知不觉的融入到血液之中。老家的“家”字，是外婆一手营造的；我们是在长一辈“家里人”三个字所包含的温情里，成长起来的。这其中，外婆的克己，勤俭，是我们家的永恒财富。雏凤凌云日，吐哺未能忘。

谨向我爱的外婆，道一声生日快乐！

Recently, one of my colleague need to scan test papers to pdf files for her girl who’s preparing middle school enrolling examination. Under Mac, Linux or any other POSIX system, it’s very easy because of the combination of 2 powerful command line tools: ImageMagick and Ghostscript.

I thought it’s easy, but it turned out need some extra works.

Conventional situation

Her images are JPEG format. Firstly, I need to convert them to pdfs by ImageMagick.

mogrify -format pdf *.jpg

Then use ghostscript to combine multiple pdfs to single pdf

gs -dBATCH -dNOPAUSE -q -sDEVICE=pdfwrite -sOutputFile=../out.pdf *.pdf

Setup tools on Windows

These 2 tools are available on windows platform. Download them from here and here.

One more thing, I need to manually add Ghostscript binary into $PATH.

Different behaviors of windows terminal

But even when I change gs to windows version gswin32c in windows cmd environment,

gswin32c -dBATCH -dNOPAUSE -q -sDEVICE=pdfwrite -sOutputFile=../out.pdf *.pdf

It still always generates error information. I noticed a post on stackoverflow.com, claimed that it may be due to the process method of wildcard differs on windows platform.

So, I need an alternative terminal other than cmd to expand wildcards. The first candidates comes to my mind is terminal shipped with Windows MSYSGit. It generally works well. Only when folders or filename contains Chinese characters, it will just exit even when I use “tab” for auto completion. I cannot cd into any directory contain Chinese files. At last, I have to install MingW,

And subsequently install mintty, to overcome the limits.

$ ming-get install mintty

The last thing

After I did so many workarounds, I found out that I could use ImageMagick merge images to pdf without the help of Ghostscript, even though someone complain the image quality is lower than original ones.

$ convert *.jpg output.pdf

• 梦回古希腊
• 贪玩的人类
• 还原金庸小说的历史真相
• 青梅煮酒话西汉：文景之治
• 中国历代政治得失
• 北洋往事：那些军阀那些人
• 密语
• 永远张国荣
• 造谣往事
• What money can’t buy: The moral limits of Markets
• 翻云覆雨

其中推荐的是，

• 钱穆的《中国历代政治得失》，对官制，财权，人事，中央与地方关系，以及封建系统的优缺点比较，非常深入，值得推荐。
• 《What Money Can’t Buy: The moral limits of Markets》是Prof. Michael Sandel在《Justice》之后，又一部对市场界限的发人深省的作品。
• 《永远张国荣》虽然是粉丝之作，难免夸大其辞，但是对他的电影算是个很好的总结。
• 《密语》描述了一个家境平凡的女孩，怎样一步一步在自尊与自卑的压力下，个人奋斗，人性扭曲的故事。
• 《女人当国》，金满楼的又一部历史普及版。金氏的书，立场稍稍与众不同，但是写法翔实，也不晦涩，既有娱乐性，也能知识性，如果带有怀疑的看，很好。
• 《北洋军阀》，也是金氏的书，条理清晰，内容翔实。相对上本，我更喜欢这个年代。乱世出英雄。

Enzyme

• rtaq:
  • < 500bp
  • will attach A to tails
• pyrobest
  • High resolution, 500-5kbp

Primer

• 0.05-1μM
• high concentration of primer may cause non-specific priming or primer-dimers.

template

• 10⁴ to 10⁷ molecules for a typical PCR
• For amplification from genomic DNA, 100 to 500ng DNA. measured by NanoDrop.
• multiplex PCR 2-5 folds amount than a typical one.

Protocols

• primer extension at 72-74°C for 1 min per kilobase of expected product.

PyQuery

In the past I used XPath to manipulate HTML tree to grab information from webpages. It simply works, but not very convenient. PyQuery is a python package which support syntax similar to jQuery. I found this package more convenient for filtering and removing nodes.

To install it:

$ pip install pyquery

I wrote a python script in ~/.bin/pyfilter.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68

#!/usr/bin/env python

import sys, pyquery
import urllib
import urllib2
import contextlib

def login(login_url,username,password):
    """Login to site and return opener.
    """
    cookietime='2592000'
    cookies = urllib2.HTTPCookieProcessor()
    opener = urllib2.build_opener(cookies)
    urllib2.install_opener(opener)
    login_form="username="+username+'&password='+password+'&cookietime='+cookietime+'&loginsubmit=true'
    page=opener.open(login_url,data=login_form)
    page.close()
    return opener

def download_text(target_url, opener):
    f=opener.open(target_url)
    u_src_txt=unicode(f.read(),'GBK')
    f.close()
    d=pyquery.PyQuery(u_src_txt)
    d=d('div.postmessage.defaultpost h2, div.t_msgfont')
    d('font[@style="font-size:0px;color:#FFF"]').remove()
    d('span[@style="display:none"]').remove()
    des_txt_set=[]
    for x in d:
        des_txt_set.append(x.text_content())
        des_txt_set.append('\r\n')
    des_txt=''.join(des_txt_set)
    return des_txt

def write_output(filename,text):
    f=open(filename,'w')
    f.write(text.encode('utf8'))
    f.close()

def main(argv):
    login_url=‘www.somesite.org/login’
    username=‘xxxx’
    password=‘yyyy’
    output='output.txt'
    opener=login(login_url,username,password)

    if ( len(argv)==2 ):
        target_url=argv[1]
        overall_text=download_text(target_url,opener)
        write_output(output,overall_text)
        return
    elif ( len(argv)==3 ):
        page_num=argv[2]
        base_url=argv[1]
        overall_text_set=[]
        for x in range(1,int(page_num)+1):
            target_url=base_url+str(x)
            des_txt=download_text(target_url,opener)
            overall_text_set.append(des_txt)
        overall_text=''.join(overall_text_set)
        write_output(output,overall_text)
        return
    else:
        print "It should be either 1 or 2 arguments."
        return

if __name__ == "__main__":
    main(sys.argv)

• As you could see, I make my script more adaptive by allowing 2 arguments or 1 argument input. so sometimes, such as online book, this script may grab all information together instead of running script for every single page.
• Something useful but I didn’t add is PyQuery.filter(), with this one, you could selectively extract text even you cannot distinguish it by class or id. For instance, you may require that all text should be longer than a length and contain or begin with some words.
• You need to pay more attention on encoding of webpage. Please see the download_text() and write_output() functions.
• Pandoc use ~ pair to generate subscript. Sometimes it may cause format trouble when converting to ePub. My solution is replace it to Chinese character ～ . They look the same, but without the side effect.

Pandoc

In the official document page, there is a chapter describing how to write an ebook by Pandoc.

I used command below to generate my epub.

$ pandoc -t epub -o book_title.epub --epub-cover-image title_image.jpg book_title.markdown

• Pandoc will look for epub.css in data-dir, which is normally in ~/.pandoc, if without specific file in --epub-stylesheet by command line.
• Pandoc generate Table-Of-Contents, but only Header 1 will be include. I used open source freeware Sigil to add header 2 into TOC. It’s easy and free of charge.

• Pandoc require blank line to separate 2 paragraphs. Strangely, even in Mac OSX line break should be “\n”, but I need to use command below in vim to generate extra line.

  :%s/$/\r/g
  

It’s a long time painful experience to write tables for markdown documents. Especially in considering so many variants of markdown, they may all have “special requirements” and need further processing.

My old solution is to use Excel, draw the format characters myself, export as tab seperated txt file and copy to the target file I need to fill the table. An interesting blog entry seems doing the same things.

Is there any good tool for that task? I googled and found something.

• Superuser.com : What tools exist for generating “ASCII Tables”, if any?
  • Asciiflow: This one is a ASCII drawing tool than a table formatter.
  • Senseful Solutions: A great tool, but can only export ASCII Art, HTML and MySQL style table.
• stackoverflow.com: Markdown editor with tables
  but the 2 projects mentioned in this post seems only parsers or compilers, not editors.
  • PHP Markdown Extra: server side markdown extension with support of table.
  • js-markdown-extra: a javascript port of the former PHP Markdown Extra. with a demo site on here

A great summary for text editor in iOS.

There’s some markdown editors in Mac OS X too:

• Commercial:
  I don’t know their supports for table yet
  • Marked: 3.99 USD
  • Byword: 9.99 USD
  • Macchiato: 19.99 USD
  • MarkMyWords: 24.99 USD
    There is some plugins to facilitate jobs like import CSV to markdown table.
  • IA Writer: 4.99 USD
• Free:
  • nvALT: A notes app instead of table generator.

The temporary soluton changes to Senseful Solutions. But I still need to modify the copied text because it’s not fully compatible to Kramdown. I check the source code, it’s based on javascript, so that I implement an online version on my own site.

My Multi-Markdown Table Generator

Please check it in my Multi-Markdown table generator.

• During the process of developing, I found that I must encapsule button tag into div tag to avoid kramdown escape button tag errorly. The warning information is:

  Found block HTML tag ‘button’ in span-level text.

• Tab in textarea didn’t produce a \t.

After evaluation of the benefits of the fusion drive, I have to maked decision which 27-inch iMac I would like to buy: iMac in Library is always taken by others especially in such a busy Mid Term Festival.

Cause I’m concerning only on the bigger 27 inch iMac. Here I would like to summarize their specifications:

• also referred to wikipedia
• optional Fusion drive (1TB mechanic hard disk plus 128GB SSD) price not available yet, but based on Mac mini’s optional price, it will cost about 250USD, roughly 2000HKD.

Pros and Cons

Pros of 2012 new iMac:

• extremely slim and new appearance. 30% lighter.
• with powerful USB 3.0 interface
• Fusion drive will increase read/write performance 3 times better.
• DDR3 1600 and maximun 32GB, old iMac only DDR3 1333 and maximun 16GB ( This limit seems not true, there are many RAM manufacturers such as OWC or Kingstone DO offer 32GB memory and it’s widely used).
• Ivy Bridge 22nm, lower heat generation.

Cons of 2012 new iMac:

• without Super Drive, my data backup need a portable optical drive, extra money.
• I’m not a intense game player, extra performance won’t help much.
• Fusion drive cause extra money, the overall price will rocket to roughly 16000HKD.
• Memory once gets into 12GB, not much improvement could be sensed.

Conclusion

I finally bought refurbished 2011 27-inch iMac, the basic configuration for 10388HKD. It’s reasonably good to me.