Martin Kou's Blog

Web Services in Python (Part 2) - Serving Static Files

2012-01-17T16:29:00.010+08:00

The last Python web server wasn't that great - it only gives you a hello world. If you're smart, you may have already figured out how to do Comet with it with very little code - we'll go over that in the next post. But.. the server doesn't even serve JavaScript files yet!

Well, don't panic. In the land of Python, everything is just a few lines of code away.

I'll assume you already have the systems libraries from my last blog post installed from this point on. If you haven't.. here's the list of things you need to install into your Linux system.

# apt-get install libevent-2.0 libevent-dev python-virtualenv

So that's 1 line of code to set up the environment - if you haven't done it already.

Next, we'll setup the sandbox for the web server - it's not absolutely needed but it's good practice to sandbox web applications with virtualenv in Python. We'll also enter the sandbox to work inside it.

$ virtualenv myapp
$ cd myapp
$ . bin/activate

3 lines of code to initialize and enter the virtualenv sandbox - not absolutely required, but good for library hygiene.

Next, we'll install a few things to our sandbox.

$ pip install gevent gunicorn pyramid

If you've got the system libraries installed correctly, it should be fully automatic - it won't even ask you a single Y/N question.

Finally, we conjure up a web server and start it.

$ pcreate -s starter myapp
$ python myapp/setup.py install
$ gunicorn_paster -w8 -kgevent --keep-alive 60 myapp/production.ini

And... and... RUMBLE... It's alive!!! (gunicorn_paster listens on 0.0.0.0:6543 by default)

Look Ma! Zero lines of Python! And it's working already!

Looking under the hood - inside the myapp/myapp/static directory of the sandbox...

martin_kou@asgard:~/Development/pyramid/myapp$ ls -al myapp/static/
total 84K
drwxrwxr-x 2 martin_kou martin_kou 4.0K 2012-01-17 00:25 .
drwxrwxr-x 4 martin_kou martin_kou 4.0K 2012-01-17 00:28 ..
-rw-rw-r-- 1 martin_kou martin_kou 1.4K 2012-01-17 00:25 favicon.ico
-rw-rw-r-- 1 martin_kou martin_kou  333 2012-01-17 00:25 footerbg.png
-rw-rw-r-- 1 martin_kou martin_kou  203 2012-01-17 00:25 headerbg.png
-rw-rw-r-- 1 martin_kou martin_kou  758 2012-01-17 00:25 ie6.css
-rw-rw-r-- 1 martin_kou martin_kou 2.8K 2012-01-17 00:25 middlebg.png
-rw-rw-r-- 1 martin_kou martin_kou 4.3K 2012-01-17 00:25 pylons.css
-rw-rw-r-- 1 martin_kou martin_kou  33K 2012-01-17 00:25 pyramid.png
-rw-rw-r-- 1 martin_kou martin_kou 6.9K 2012-01-17 00:25 pyramid-small.png
-rw-rw-r-- 1 martin_kou martin_kou   49 2012-01-17 00:25 transparent.gif

.. and that's where you can put up your favorite JavaScript, CSS and HTML files. Have fun!

2011 in a one word for me...

2011-12-31T22:20:00.001+08:00

Betrayed.

Web Services in Python (Part 1) - You Only Need 3 Lines Of Python Code

2011-12-29T22:22:00.020+08:00

I cringe whenever I see someone meddling with Apache or nginx config files to do something simple these days.

And worse - the usual LAMP stack is terrible at handling anything that's actually asynchronous in nature, like WebSocket or Comet. It can also be a pretty big security hole as well, see Slowloris.

And yet, you have to read lengthy manuals and fiddle with lengthy config files to get anything done with the traditional web servers - instead of getting shit done, like, right now. Seriously, this is 2011. You're doing more for less if you're sticking to the 2000 ways.

Let's try another way.

So, let's say you have a usual Ubuntu, or Debian, or the more trendy Mint machine. Try this in your home directory:

$ sudo apt-get install libevent-2.0 libevent-dev python-virtualenv
$ virtualenv webservice
$ . webservice/bin/activate
$ pip install gevent gunicorn

Well, congratulations! You've already got a self-contained web server package installed with the above four lines of shell commands! If your Linux machine has libevent and virtualenv installed already, you can even cut out the first line.

So, 3 lines of shell code to get the infrastructure ready.

The Apache guy is.. eh.. still looking through the manual pages for how to set up virtual hosts. We'll get back to him later, after the late-night news, supper, and.. watching grass grow.

Now, for the web server's content, we'll try a Hello World first. It'll be a Python script for now - we'll make it more like a real web server (i.e. serves your .html, .js, .css and runs scripts and frameworks just like your Apache, nginx, etc.) in another post.

$ cd webservice
$ cat > application.py
#!/usr/bin/env python
def application(environ, start_response):
 start_response("200 OK", [("Content-type", "text/plain")])
 return [ "Hello World!" ]

Just in case you're not familiar with UNIX, press Ctrl-D after the last line to save the file.

So, 3 lines of Python code to make the server serve.. something.

Finally.. we start the web server:

$ gunicorn -w8 -k gevent --keep-alive 60 application:application

And.. it's alive! It's running at port 8000 by default. You can see it in http://localhost:8000/

The performance isn't shabby as well, considering the web server part of it (i.e. gevent.pywsgi, which handles the underlying HTTP protocol) is written in 100% Python:

$ ab -c 32 -n 12000 http://localhost:8000/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 1200 requests
Completed 2400 requests
Completed 3600 requests
Completed 4800 requests
Completed 6000 requests
Completed 7200 requests
Completed 8400 requests
Completed 9600 requests
Completed 10800 requests
Completed 12000 requests
Finished 12000 requests


Server Software:        gunicorn/0.13.4
Server Hostname:        localhost
Server Port:            8000

Document Path:          /
Document Length:        12 bytes

Concurrency Level:      32
Time taken for tests:   1.182 seconds
Complete requests:      12000
Failed requests:        0
Write errors:           0
Total transferred:      1644000 bytes
HTML transferred:       144000 bytes
Requests per second:    10148.61 [#/sec] (mean)
Time per request:       3.153 [ms] (mean)
Time per request:       0.099 [ms] (mean, across all concurrent requests)
Transfer rate:          1357.77 [Kbytes/sec] received

Connection Times (ms)
        min  mean[+/-sd] median   max
Connect:        0    0   0.2      0       4
Processing:     0    3   1.9      2      22
Waiting:        0    3   1.9      2      22
Total:          1    3   2.0      3      23

Percentage of the requests served within a certain time (ms)
50%      3
66%      3
75%      4
80%      4
90%      5
95%      6
98%      8
99%     11
100%     23 (longest request)

For comparison, Apache 2.2 running on the same machine, serving a static file does 12k requests per second. And you can pretty much forget about playing with Comets in it without a lot of configurations and modifications.

I bought a domain at Namecheap today, because GoDaddy supported SOPA

2011-12-26T17:28:00.007+08:00

It's martinkou.com - nothing special, just a domain for my own computers.

I haven't even set up a web server on it yet.. but it's a static IP and it's linked to a small Linux machine at my home.

If you think having a Great Firewall in the US via SOPA is a bad idea, I highly encourage you to do the same or something similar. It's only $12 with the SSL certificate. You can also pass this little message around if you want your friends to do the same.

If you want something other than Namecheap for some reason, Lifehacker has a list of alternatives to GoDaddy. There's also a list of GoDaddy-owned registrars that you should avoid at nwlinux.

The Easy Way Out

2011-11-28T18:22:00.019+08:00

It's not easy these days to find people who can talk in a way that I feel truly interesting, but the ski trip I had this weekend did lead me to such people.

X: "I feel America is on the way to decline, we're here to witness it... This will be like what happened to the British Empire in the last century. This century is going to be the China century."

Me: "I don't think so, China's legal system is a mess. We (as in the Chinese people) still has some very big obstacles in front of us."

It's funny when you have one US native and one Chinese native both thinking their respective countries are a mess.

But I neglected to tell him the details.

The Easy Way Out

A few months ago, I was sitting in Cheesecake Factory in Palo Alto, talking to an entrepreneur from Stanford. It's interesting how the same topic often happens with Americans.

Me: "The mindset of the typical mainland Chinese is very different from the people here (Silicon Valley), and it will be for a long time, I think. It's hard to see what really matters when you see corruption all around you. You'd simply lose your purpose and direction."

Y: "I know what you're talking about. I lived in Mexico. People think they can get ahead by playing little power games here and there. Nothing real gets done in an environment like that."

Me: "Yup, taking the easy way out, instead of innovating.. that's what I meant."

The conversation ended in a pretty sobering conclusion - if the future belongs to one of the existing countries of the world, it'll be neither the China nor America that we see today. Either they change a great deal, or another country will take up the position.

I hope it's a China 2.0... But, good way to spot sinking ships there.

Salesmen and Bean Counters

On the return from the ski trip, I read this.

“The company does a great job, innovates and becomes a monopoly or close to it in some field, and then the quality of the product becomes less important. The company starts valuing the great salesman, because they’re the ones who can move the needle on revenues.” - this one is actually from Steve Jobs. But can be found in the article.

It (the article, not SJ) continues, "... It’s not just the salesmen. It’s also the accountants and the money men who search the firm high and low to find new and ingenious ways to cut costs ... The activities of these people further dispirit the creators ... crimp the firm’s ability to add value to its customers. But because the accountants appear to be adding to the firm’s short-term profitability, as a class they are also celebrated and well-rewarded, even as their activities systematically kill the firm’s future."

"In this mode, the firm is basically playing defense. Because it’s easier to milk the cash cow than to add new value, the firm not only stops playing offense: it even forgets how to play offense. The firm starts to die."

With all due respect, I respect salesmen and accountants when they actually pay attention to the details - not the numbers after the decimal point, but how things went from point A to point B and how it can go to point C or D. Warren Buffet greatly values accounting skills when properly used to assess a company's underlying value, for example.

But then, I think we all know what happens to companies who simply focus on their financial reports, or their "VC relations" for startups - rather than their customers. So while I don't believe everything the article says, it has something to keep in mind.

Ethics

This is my favorite.. I love playing this with some people and see how they react. Made plenty of people hate me. But if your brain is hardwired like that, you're not gonna be my (or any hardcore entrepreneur's, or hacker's) friend anyway.

Z: "I have this assignment which is about what kinds of medicines would be the most useful to a third world country, but are not yet available to them."

Me (after reading a bit into the report): "I don't see the logic in your conclusions."

...

Me (after a lot of discussion): "Ok, we're not going anywhere with all these different concerns about the kinds of sickness in a third world country, the affordability of the medicines to the people, the people's life expectancy, age structure, etc. How about this... If the new medicines are actually used, people (or their government) would need to buy them. We can then use market demand, and thus, money, as a proxy measure. So, what kind of medicines do you think could be the most profitable if made available to the poor country?"

Z: "All that you think about, is money. (To some other guy) I'm trying to save lives here, do you think it's correct to think about money?"

It's useless to explain anything to Z after this point - if you don't want to accept an opinion, labeling the source as "evil" is the easiest way out. To a computer scientist, it would be totally intuitive - to sort a list (most useful medicines to a poor country), you need to have a single metric. You can get people to hate you for telling them this. You can also get people to love you for totally revolting concepts with other trigger words. Looking for cracks in people's minds - salesmanship. Find a friend to practice near you.

Present State of HTML5 in Mobile App Development

2011-06-11T16:30:00.011+08:00

“We’re doing our next mobile product in HTML5.”

“Yeah. A lot of people do that these days. I’ve been playing with Appcelerator as well.”

“Uh.. but that’s not what I meant by HTML5.”

I’ve been having a lot of this kind of conversations lately - partly because I’m doing an HTML5 app myself. But like “AJAX” back in 2005, the term “HTML5” is never clearly defined and is quite easy to throw around without considering the actual technical benefits, the trade offs, and the implications in the operations side. If you’re working in a company managed by buzzwords of the day, then Dilbert may very well be sitting by the next cubicle - if you’re lucky.

Two Divergent Roads

When people talk about HTML5 on mobile devices, there are actually two very different things that they may be talking about.

The more intuitive, and technically simpler path is to use the web browser - either embedded to your app (e.g. PhoneGap) or, the app is supposed to be opened from a bookmark icon on your mobile phone (e.g. Financial Times for iPhone and iPad). The usual benefits of this path include being able to reuse existing web design and development talents, products are highly and trivially portable, are easy to debug and fix (e.g. via weinre), and thus, fast iteration. Trade offs are performance and features - features is less of a hassle if you use an embedding architecture like PhoneGap. Performance is the #1 problem for this path.

The not-so-intuitive way is to use the JavaScript engine to control native features directly, ignoring the web browser component on mobile devices. Performance problems in HTML5 apps are mostly caused by HTML/CSS rendering, not JavaScript execution - so this approach can and do give you massively increased performance - if used correctly. Examples of this approach include Appcelerator Titanium, Mobage/ngcore from ngmoco, Game Closure, and whatever PhobosLabs is doing.

Node.js for Client Side

Take PhobosLabs’s project as an example - what they’ve done is take WebKit’s JavaScriptCore component, interface that to OpenGL on the device’s side, and expose an HTML5 canvas API on the developer’s side. What this means is, the developer can be developing and testing his game on a desktop browser with good canvas support, and put that to his mobile device and have his game run with similarly good (and perhaps better-than-desktop) performance. This is very similar to the approach taken by Node.js - take the JavaScript engine and add your own stuff to it for your specific use case.

Appcelerator Titanium expands on that concept and exposes a whole abstraction layer of UI widgets - such that it can be applied to general applications in addition to (instead of?) games. What this means is, an app developer can create buttons from Appcelerator’s UI library in JavaScript, and Appcelerator’s internal logic will translate that to e.g. a native UIButton on iOS. The UIButton on screen will then be completely controllable within JavaScript - the developer, theoretically, won’t need to write any Objective-C.

The trade off to this kind of approach is, while you’re still coding in JavaScript, you’re saying goodbye to the layout logic and declarative style definitions given by HTML and CSS. You’re also saying goodbye to the excellent debugging tools available to normal HTML5 developers. This is a smaller problem, and thus a sensible trade off, for HTML5 gaming APIs like Mobage because canvas exposes a relatively small set of interfaces - so there are less things to go wrong in the first place. But once the interfacing layer gets big, like in Appcelerator’s case - if anything goes wrong, you’d have to go back to Objective-C (or Android API)’s level, and also take the additional complexities added by the interfacing layer into account - while debugging. A lot of the negative reviews for Appcelerator out there can be understood if you take this in mind.

Back to the Browser

The first problem of doing a full-stack HTML5 mobile app is that it’s slow. The second problem is that the tool chains (e.g. broken persistent nav bars in jQuery Mobile, iOS’s innerHTML bug I discussed earlier) out there are still very buggy - which either means you need to reduce your features to avoid the bugs or you spend some time to work around the bugs. Features is only really the third problem - you can get around that with a plugin architecture like PhoneGap.

If you did some experiments yourself with one or two UIWebViews in an iOS app, plus a bit of your own JavaScript - no non-essential libraries added - you’ll find that it’s quite feasible to make a fast, full-stack HTML5 app... With no features. The PhoneGap iOS project template takes only 1 to 2 seconds to launch on an iPhone 3GS - a relatively old phone. What that tells you is, as a baseline, full-stack HTML5 apps can be very fast. So, if you find out your HTML5 app is hanging 10-15 seconds for some operations, or the app is taking 15 seconds to launch - then it’s almost always some JavaScript or UI library’s fault.

Two UI Libraries

What a mobile full-stack HTML5 app developer needs is usually two things: one, a bridging layer between the native platform and the web view; two, a mobile UI library. PhoneGap is pretty much the default choice as the bridging layer these days - it allows you to do things like calling out the camera, accessing the address book, reading and writing files, from JavaScript. The two most popular UI libraries are jQuery Mobile and Sencha Touch.

jQuery Mobile is a project founded late last year so it’s relatively new. It is quite obviously immature to anyone having actually tested it - persistent navigation bars are broken, page switching is noticeably slower than native, no way to incrementally add to lists without refreshing the whole page, high CPU usage when tested on desktop platforms (as of alpha 4), etc. I chose it for my project for its relative simplicity (i.e. hackability), the MIT license, and because it’s based on jQuery - every respectable web developer knows jQuery so it’s easy to train someone in jQuery Mobile.

Sencha Touch is, purportedly, much more mature and is faster. I have an instinctive abhorrence towards things that comes with a higher complexity, because I always have a feeling that there are gonna be a lot of features that I wouldn’t use and end up as dead weights pulling down the overall performance. I may be wrong though - the top featured mobile app on PhoneGap’s apps page is IGN Dominate - it’s extremely fast and it’s based on Sencha Touch - although I’m sure they’ve put in a lot of their own optimizations to do that.

Debugging and Making Changes

But the upside of doing full-stack mobile HTML5 - which a lot of people misses - is that it’s easy to debug and make changes. Any developer who’s worked on any non-trivial project would tell you, debugging and maintaining the project can easily take up 80% of the time, if not more. What this means is, when you see a certain tool chain claiming you can build, e.g. a chat app, in 15 minutes - it’s really telling you it has compressed 20% of your trouble into 15 minutes - the remaining 80% may have been made 3x worse.

This was a tough problem for mobile HTML5 apps because console.log() in a UIWebView normally logs to nowhere. So if anything goes wrong in JavaScript, you would need alert() or you wouldn’t be able to see that. PhoneGap improved this a bit by patching in a console.log() that logs to XCode’s console. But that’s still not enough.

The best solution right now is weinre. It is buggy and slow but it works - with that you can literally debug your mobile app’s UI from Web Inspector on your computer. It is based off WebKit’s Web Inspector with its debugger back end taken out and replaced with a remote back end. I’ve done some investigations into Web Inspector’s code myself two or three weekends ago and found that it’s not really that hard to turn it into a remote debugger. weinre’s development may speed up in the coming months or someone may make a better project replacing it - we’ll see.

This will be the actual big thing in mobile app development for the coming years, because it solves tons of problems in the 80% part of the app developer’s workload. Want to change your UI design in Objective-C? Edit, recompile, run - repeat until you get satisfactory results. This can easily take a whole day if the recompile step is long. Doing that same thing with HTML5? Edit a few CSS attributes in weinre and experiment away - you don’t even need to shut down the app. Yes you can debug an HTML5 mobile app, to some extent, by running it in a desktop browser. But believe me - your app will have a bunch of bugs that happens only on the mobile device, so weinre is absolutely needed.

Unfortunately, people usually tout their toolkits and their features and seldom glorify useful debuggers, so I guess it won’t get talked a lot even though it’ll actually be the thing that mobile HTML5 developers use the most.

The Present State

I hate writing conclusions, but this is a really long blog post, which even myself will need a few points to remind myself - when I read my own posts later.

Two ways to do HTML5 apps on mobile, full-stack or only using the JavaScript engine.
The JS engine approach makes sense for making games. I have reservations for implementing a full UI toolkit on that due to the complexities involved in the interfacing layer - but, we’ll see.
Full-stack HTML5 mobile app is slow and buggy, but can be mitigated with custom optimizations. Not many people have managed to do that but it's possible.
The real benefit of doing HTML5 mobile apps is the possibility of debugging and changing it online. Native app developers will have a hard time matching that kind of development speed advantage - once the team has problem #3 resolved.
Oh yes, porting is easy as well. But I assume everyone gets that automatically.

How to make the Push Notification prompt reappear during iOS app development

2011-06-02T07:38:00.004+08:00

If you're developing an iOS application and your application is using push notifications, chance is, you'd like to be able to test the push notification registration step during debugging.

Problem: the push notification dialog only appears once for your debug application. It doesn't matter that you remove the app and reinstalled it - it won't appear again. This kind of behavior is dangerous for developers because it means it's impossible for you to test the push notification registration step reliably - it might have worked last week but stopped working this week, and you just wouldn't know.

Apple has a technical note dealing with this behavior. What an app developer can do in this case is to uninstall the app and adjust his development phone's clock to 1 day later. If he reinstalls the app after this, the push notification dialog will be allowed to appear again.

But... there's always a better way. It requires jailbreaking your iPhone but who doesn't? :)

Now for something completely different...

2011-05-28T20:10:00.008+08:00

Another midnight/early morning note to my blog so that I don't forget what went in my mind earlier.

The current approach of embedding scripts for all browser support in libraries like jQuery, and embedding all available widgets in libraries like jQuery UI and jQuery Mobile - is completely wrong. The extra parsing and code complexity may not matter for Core i7 desktops, but it really hurts performance in mobile devices.
If the uncompressed size of the .js file of the UI framework for your upcoming mobile app is 1.5MB+ ... you're doing something very wrong.
Narrative JS. Can be used to solve a lot of problems if it works well. I should find some time to investigate this thing.
The current approach PhoneGap is using for communications between JavaScript and Objective-C is very hacky and slow - even though it works. Adding a micro Comet server on Objective-C's side and use Comet for communications will work much better. It sounds crazy, but it's more doable than most people think.
I need to do more screencasts like AFeature's, for those "micro" open source projects (like bson and bson-objc) that I do in weekends - just to brush up my presentation skills.
The current way people do Comet and WebSockets haven't accounted for unreliability at the physical layer on mobile devices (aka. AT&T).
Comet on mobile devices is also not a very good practical idea because (from our experiments), mobile browsers tend to minimize active TCP connections - s.t. even though you can sort of keep the receiving connection "persistent", the send will not. Which means, the send part of your Comet session will have high latency.

Some predictions and wishlist for the coming year

2011-05-22T04:12:00.004+08:00

It's a bit late to do it in May but I'm beginning to find myself constantly forgetting big picture things.

Mobile applications will increasingly be done in HTML5 and wrapped inside something like PhoneGap.
Co-routine I/O libraries like gevent and eventlet, and the whole approach in general irrespective of languages, will take off. Someone will implement an Actor model on top of them to make them work nicely across processes. (note: Pykka doesn't work across processes, yet)
JavaScript.next will get more buzz.
CoffeeScript will get a lot more users, especially for those people on Node.js. Writing sophisticated server-side logic in plain JavaScript is crazy.
NowJS will be a flop.
ChromeOS will be a flop - for now. But Google has the money and patience to iterate.
WebSockets and WebGL will remain experimental.
The general direction for software that'll take off are things that are lean and mean - in terms of performance, code size, ease of development, etc.
Everybody that matters will will say UX instead of UI - which means just a good UX will no longer confer significant competitive advantage in general.
The HK government will keep pushing meaningless things like "the wine industry" where a single tech company will easily trump the whole industry in terms of revenue - but they'll keep talking the talk on tech.
HK university graduates still don't know their mainland counterparts from similarly high-ranked universities are already getting a higher salary and maybe 2x to 3x their real purchasing power.
The world will not end tonight, doomsday preachers are nuts.

An alternative workaround for Mobile Safari's innerHTML problem

2011-05-21T07:49:00.004+08:00

One major frustration for mobile HTML5 app developers is that innerHTML in Mobile Safari stops working randomly - often at the most inconvenient times. I met the problem as the PhoneGap project scaled up in complexity in the recent months and random rendering bugs appear more and more often in my app.

There's a clear problem description in this 4-year-old blog page:

The code has been in use for several months without issue and has been working fine all this time in all the browsers I’ve tried it in. For some reason in this one particular scenario though, Safari was completely ignoring the attempts to set the innerHTML of the node. Setting the innerHTML and then on the following line attempting to read it was also giving an empty response. For example:

text = "foo";
node.innerHTML = text;
alert( "html="+node.innerHTML ); // Pops up message saying "html="

I tried numerous methods to fix this problem, including setting the property before and/or after adding the DOM node to the document. I also googled it which flagged up a number of related posts but these generally referred to pages that were served as XHTML (ie. pages with an .xhtml extension and MIME type application/xhtml+xml) and were not offering any solution.

The general consensus towards working around this problem is to set a timer to retry setting the innerHTML value again. (e.g. as mentioned in this page) Such a solution is highly inconvenient and is generally inapplicable, however, because it screws up your programming model - you essentially need to turn the simple operation of setting innerHTML into a callback-like asynchronous operation.

A Probable General Solution

I found an alternative solution yesterday when I noticed that, at the moment innerHTML stops working in Mobile Safari, DOM manipulation operations, like..

element.appendChild(document.createTextNode("Baby you're a firework!"));

.. would still work. What this means is, if you're able to translate the set innerHTML operation into purely DOM manipulation operations, you'd be able to circumvent the problem entirely - in theory.

To do this, I'd need to be able to parse the HTML in JavaScript. That's actually easier than most people think - I've worked with such a parser during my CKEditor days, and it was mostly just a bunch of regular expressions. Even better, there's a standalone JavaScript library for doing HTML parsing written by John Resig.

With that in mind, I wrote a simple JavaScript function that turns an HTML string into a DOM fragment - which in mathematical terms, is simply a forest.

Then, as I'm using jQuery Mobile, I can patch it into jQuery's html function:

Done!

AFeature - Interactive JavaScript console for mobile HTML5 applications

2011-05-18T14:29:00.004+08:00

I created this because I really needed something interactive to debug my PhoneGap iOS project. There is a similar project out there called ibug but it is no longer working.

You can get the code here, and below is a quick screencast.

Edit: Turns out it was unnecessary. I kept searching for Mobile Firebug before I came to the conclusion that I need to write one myself. But someone from the PhoneGap user group told me there's actually a Mobile Web Inspector out there. It's good enough for me for now, although I have to say I don't really like the Java server code in there... looks overly complicated to me.

How to solve "assembler for architecture ppc not installed" errors after installing XCode4

2011-05-09T07:50:00.003+08:00

I was upgrading and building a few Python modules on my Mac this afternoon, and I got this error:

Martin-Kous-MacBook-Pro:~ martinkou$ sudo easy_install -U Cython
Searching for Cython
Reading http://pypi.python.org/simple/Cython/
Reading http://cython.org
Reading http://www.cython.org
Best match: Cython 0.14.1
Downloading http://cython.org/release/Cython-0.14.1.zip
Processing Cython-0.14.1.zip
Running Cython-0.14.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-5NQPRk/Cython-0.14.1/egg-dist-tmp-gOLK4r
warning: manifest_maker: MANIFEST.in, line 19: 'recursive-include' expects <dir> %lt;pattern1%gt; %lt;pattern2%gt; ...
/usr/libexec/gcc/powerpc-apple-darwin10/4.2.1/as: assembler (/usr/bin/../libexec/gcc/darwin/ppc/as or /usr/bin/../local/libexec/gcc/darwin/ppc/as) for architecture ppc not installed
Installed assemblers are:
/usr/bin/../libexec/gcc/darwin/x86_64/as for architecture x86_64
/usr/bin/../libexec/gcc/darwin/i386/as for architecture i386
/tmp/easy_install-5NQPRk/Cython-0.14.1/Cython/Plex/Scanners.c:6694: fatal error: error writing to -: Broken pipe
compilation terminated.
lipo: can't open input file: /var/tmp//ccAtkL0J.out (No such file or directory)
error: Setup script exited with error: command 'gcc-4.2' failed with exit status 1
</pattern2></pattern1></dir>

This error pops up for a lot of build scripts besides Python distutil's - Perl modules like ImageMagik are also failing to build on OSX after Xcode4 is installed. The reason for this is Apple has removed the PPC assembler for OSX platforms in Xcode4, yet a lot of existing build scripts for OSX are building universal binaries.

There're quite a few solutions on the Internet, but the cleanest solution I've found so far, is the second solution (as of today) in this Stack Overflow page.

So it turns out Apple has still left a PPC assembler in the iOS platform files for Xcode 4. All you need to do is to make 2 sym links at the appropriate place and it'll work.

$ sudo ln -s /Developer/Platforms/iPhoneOS.platform/Developer/usr/libexec/gcc/darwin/ppc /Developer/usr/libexec/gcc/darwin
$ sudo ln -s /Developer/Platforms/iPhoneOS.platform/Developer/usr/libexec/gcc/darwin/ppc /usr/libexec/gcc/darwin

矽巷啁啾：朱克伯格以外的事（三）

2011-05-09T07:47:00.001+08:00

互聯網站 AirBnB在 SXSW後雖然確立了宣傳手法和自我特色，但仍然是小型搞作，每次只是供應幾十間住房，談不上賺錢。而談不上賺錢，對於創業者是大問題，因為幾個創辦人在三藩市的日常開支，絕不便宜。

08年夏季是美國總統大選的年頭。想一次過滿足三個願望：賺生活費、幫自己賣廣告，和令自己可以在全國電視出現，可以怎做？

AirBnB三人靈機一觸，由早餐想到穀物片，由穀物片想到穀物片背後的品牌和廣告，再由穀物片品牌想到總統大選。於是美國的電視上就出現了：「 Obama O’s： Hope in every bowl！」及「 Cap’n McCain’s： Put a maverick in your morning！」

利用網上宣傳的便利，美國人對總統大選的瘋狂，和前兩次搞 AirBnB網站所認識的傳統媒體， AirBnB推出了以奧巴馬和麥凱恩為主題的早餐穀物片。每款限量推出 500盒，每盒售價 40美元。 1000盒穀物片送 200盒給記者。其餘 800盒讓 AirBnB的名字除在美國迅速走紅之外，還幫 AirBnB賺了三萬美元。

「我對你們的計劃信心不大。」 08年尾， AirBnB三人去了考 Y-Combinator。考 Y-Combinator的原因並不是因為 AirBnB成功，而是因為他們沒錢。賣穀物片的 3萬元只是還了部份信用卡欠款。在全國電視一輪風光過後， AirBnB網站依然沒人流，沒收入，很快便被人忘記。折騰了一年，結果碌爆信用卡，晚餐只好吃麥凱恩和奧巴馬。考 Y-Combinator，就是想問 Paul Graham拿兩萬元給自己吃飯。

「但我欣賞你們那種大無畏的精神。不怕死，能在最壞的情況下看出並抓緊機會，比任何計劃書更好。」 Paul Graham由 AirBnB網站的統計數字觀察到，除大型會議以外， AirBnB的主要客源來自紐約。於是， Paul Graham給 AirBnB的頭號建議是，不要老是坐在矽谷，去紐約見你們的客戶。打電話給他們，敲他們的門，和他們一起拍照、開派對，用盡所有方法，令客戶記得你。

「不要想，只管做」，紐約一行終令 AirBnB人流增長。

(刊於 2011-02-26 香港蘋果日報)

矽巷啁啾：朱克伯格以外的事（二）

2011-02-17T05:20:00.002+08:00

AirBnB的故事在 07年 10月開始，那是 IDSA（ Industrial Designers Society of America）在三藩市的大型展覽會前夕。創辦人之一的 Brian Chesky辭掉原有的工作，搬到三藩市和朋友 Joe Gebbia一起居住。目的是到矽谷附近的三藩市，嚐嚐創業的滋味。

當時三藩市的酒店都因為展覽會全部爆滿。 Joe和 Brian覺得，這個時候出租家裏多餘的空間給短期住客，應該有市場。二人建一個簡單的網站，讓附近的人可以出租家裏的空間給參加展覽會的短期旅客，當中包括 Joe和 Brian家裏的三張氣墊床。網站一做好，兩人就以電郵主動聯絡一批與工業設計有關的博客。

不消幾天，一些和展覽會主題有關的大網站，就出現了有關 AirBnB的報道。 Joe和 Brian也如此幫自己找到 3個短期租客，賺了約 1000美元的租金。更重要是兩人體驗到短期租務市場的潛力，確立了日後透過社交媒體宣傳方向。

展覽會結束， Nathan Blecharczyk加入 AirBnB。他們用同樣的方法，攻打 08年 SXSW藝術節。宣傳同樣由博客網站入手，先找小博客，報道過就找多點人看的博客，報道一輪就聯絡更多人看的網站。如是者， AirBnB早期的宣傳方法便成形。

吃自己狗糧　了解客戶感受

那時的 AirBnB還有一個問題，要找住房，其實早就有 Craigslist和 Couch Surfing。 AirBnB當時的報道雖多，但只得幾十間住房，要擴張就得找個令自己與別不同的地方。

Joe和 Brian在這個時候，用自己的網站租了一間房來住，目的是了解客戶用 AirBnB租房整個過程的感受。矽谷稱這種做法做吃自己的狗糧。

這樣租間房，去時和房東談得投契。到了吃飯，氣氛還是融洽，但很容易就談到租金。一談租金，吃飯變成買賣，雙方就由結識新朋友變成擔心對方的信用。一下子，整個旅行、租房、看藝術節給人的感覺變得很凉薄。

這樣吃一吃狗糧， AirBnB團隊學到要避免在租住過程中談到租金， AirBnB就得要求租客先行付款。如此房東可以更放心放租，租客亦不會再受租金困擾， AirBnB亦可以從中收取佣金。比別人優勝的用戶體驗，成為 AirBnB的特色。

(刊於 2011-01-29 香港蘋果日報)

Some performance tests on gevent and libevent latencies

2011-02-15T08:39:00.010+08:00

I set out to test the bare minimum reaction times for gevent and its underlying libevent library this morning. The exact versions used are gevent 0.13.1 and libevent 1.4.2 - not the most updated but fairly common.

The testing method I used was to run a TCP client and a server in the same process and let them ping-pong each other, and measure the rate of ping-pong requests. What this means is basically the absolute best reaction times for gevent and libevent in terms of TCP socket events. As a control experiment I also did a gevent queue ping-pong between two greenlets.

The source code of the two simple tests can be found below:

libevent test case:
http://tixxme.com/~martinkou-public/benchmark.c
gevent test case:
http://tixxme.com/~martinkou-public/benchmark.py

And here are the results -

Macbook Pro 2007 (2.2GHz Core 2 Duo/4GB RAM/OSX 10.6.6)

Martin-Kous-MacBook-Pro:testgswitch martinkou$ ./benchmark
25441.49 roundtrips per second
Martin-Kous-MacBook-Pro:testgswitch martinkou$ ./benchmark.py
test_tcp_socket: 8722.40 roundtrips per second
test_queue: 265788.63 roundtrips per second

Xeon Server (Xeon X3450/8GB RAM/Ubuntu Server 10.04)

martinkou@hydrogen:~/Development/levent-test$ ./benchmark
105852.88 roundtrips per second
martinkou@hydrogen:~/Development/levent-test$ ./benchmark.py
test_tcp_socket: 14124.71 roundtrips per second
test_queue: 430830.17 roundtrips per second

So, despite gevent's awesome performance among all Python concurrent I/O libraries - it still has plenty of space for improvement. This is actually a pretty good news for the developers of gevent and similar concurrent I/O libraries in Python. Because it means the more awesome things are still to come.

矽巷啁啾：朱克伯格以外的事

2011-01-18T01:18:00.005+08:00

facebook新年市值衝上 500億美元，一間 80後開的公司市值高過港股火車頭，這是現在不少人對科技的理解：錢。但之前講過，在矽谷創業及投資的人非常多。在朱克伯格以外，還有很多故事。
「我們要做一個網站，讓家裏有多餘空間的人，租給短期旅客，令那些 backpackers之類的人有便宜的食宿選擇。」
「但這個市場不是很小嗎？萬一發生了強姦搶劫之類的事，你們怎麼辦？」
問以上問題的是 Y-Combinator創辦人 Paul Graham。他說你的腦筋有問題，很明顯你要見心理醫生。

創業者常有荒誕思維

作為創業者，願景荒謬是常見的事。甚麼 Google或 Apple，現在馬後炮說他們聰明很容易，但在他們開始的時候，其實都很荒唐。七、八十年代走去史丹福 Homebrew Computer Club的人，不少是長髮長鬍子，滿身臭味的嬉皮士，有些甚至會吸大麻。那時候的微型電腦，只是一個盒，沒螢幕沒鍵盤，只有一排排的開關和燈。即使是那班嬉皮士，也不知道有甚麼用。
在那個時候，如果你撞到滿面鬍子的喬布斯，你會叫你的兒子不要學他，然後叫你的女兒千萬不要嫁這樣的人。
話說回頭，想做那個網站的人，很明顯是儍的。剛巧， Paul Graham都是儍的，所以， AirBnB這個團隊考進了 Y-Combinator。

AirBnB碌爆卡才成功

AirBnB並不是考入 Y-Combinator才開始。 08年的 airbnb.com，是 AirBnB團隊第三次搞同類型的網站，前兩次都是失敗的。正常人的反應是放棄，一般人根本不會試三次，或失敗一次就放棄。
三個人為了試第三次，碌爆信用卡。但今時今日， AirBnB在美國的影響力足以令傳統酒店業震驚。單是紐約， AirBnB可找到的單人住房已超過 4000間，而且不限於低檔住房。 AirBnB的住房現已遍佈 168個國家， 8000多個城市。 2010年 11月， AirBnB向 Sequoia Capital和 Greylock Partners籌了 720萬美元的 Series A投資資金。
這類故事在矽谷多次發生。第一、二次人們會覺得神奇。幾十年來屢屢發生，是因為人們會隨着時間學習，發展出系統。在每一個 AirBnB疑似儍呆子變老闆神話的背後，其實都有一個實在而合理的故事。下回再講 AirBnB的故事。

(刊於 2011-01-15 香港蘋果日報)

A Review of 2010

2011-01-01T00:32:00.007+08:00

So, here's my story so far, in short...

Joined a high-ranking open source project, FCKeditor as Core Developer (2007)
Made it to computer mags (2008)
Made it to newspapers (2009)
Made it to TV (2009)
Went to Beijing and caught a glimpse of the central government's ambition with the high tech industry in China (2009)
Traveled to Silicon Valley and lived for six months (2010)
Made it into two Silicon Valley incubators almost simultaneously and accepted one of the offers - SSE Labs (2010)
Produced two prototype demos within the six months in SV, one of them was made public (SayCheeze), demo-ed to VCs. (2010)
The other idea is more promising... so I'm working on that ;)
Hooked up to an editor from Apple Daily HK which allows me to write column articles in weekends. (2010)

So I've seen plenty of action in 2010. The people I've seen were almost a blur - the top VC partners and angels (i.e. principals from funds like August Capital, Sequoia, KPMG, etc. and then people like Paul Graham - yes, I've talked to Paul Graham, in person), the Sandbox network, met a number of SV entrepreneurs from having IPO'd to having got significant funding to having sold their stake for good like RockYou's Jia, Webshots' and Stumbled Upons' founders... I've even talked to someone affiliated with the White House for a good 45 minutes.

It's quite surprising what a determined enough person can do in 3 years. But still, I'm very far from Mark Zuckerberg...

... and that kinda puts in perspective, how big this world is. Mark is only 1 year older than me. A Forbes ranking is not too significant in my eyes - plenty of people can get insanely rich by being lucky - but a hand that guides the path of technology and thus, civilization, is significant. And I mean all these in a concrete, pragmatic sense, in case you think I'm just another dreamer.

Having seen all these powerful people, the body of knowledge and business practices they've built, reminds me of a quote from H2G2:

Space is big. You just won't believe how vastly, hugely, mind- bogglingly big it is. I mean, you may think it's a long way down the road to the chemist's, but that's just peanuts to space. - Douglas Adams

矽巷啁啾：矽谷的融資環境（二）

2011-01-01T00:29:00.002+08:00

「天使投資者」之後一層就是「風險投資」。風險投資通常以私募基金形式運作，由之前有成功投資或創業經驗的人領頭，向不同類型的投資者如金融機構融資，再投向新的科技企業。

風險投資對於公司估值、投放資金等變化較大，少的可以投資數十萬美元，大規模的動輒投資過億美元，但仍有一定框架。

譬如一間拿過天使投資，開始做出產品雛形，有合理盈利模式，但之前團隊沒有甚麼往績的新公司，剛開始找風險投資時，通常會拿到一個 Seed Round或者 Series A。這時公司的估值通常是數百萬至一千萬美元， Seed Round投入的資金，通常是幾十萬至一百萬美元，而 Series A則由一百至幾百萬美元。

種子資金提供生存時間

Seed Round和 Series A的作用是，給公司一至兩年生存時間，在產品開發上和市場上建立相當實力。 Series A之後的科技公司會有很多不同去路，需要再擴張，可繼續籌 Series B、 C等，這些時候投入的資本，通常會過千萬甚至逾億美元。

不過繼續融資，在為公司帶來大量流動資金的同時，股份會被攤薄，外來投資者所佔股份很可能會過半，創辦團隊需面對有機會失去公司控制權的風險。

而且在股份被大幅攤薄下，即使公司後來被收購或上市，但創辦團隊可以分享到的財富可能會變得很少，較一般管理人員一年薪水更少的情況亦偶有發生。所以，向風險投資基金籌錢，並不一定越多越好。

另一個可能性是，被較大的企業如 Google、 Apple、 Facebook或 Amazon等收購，價錢視乎情況。近年大型企業收購小型科技公司例子越來越多，金額通常在數百萬至數千萬美元。

藉收購、擴市場網羅人才

大公司收購小型科技公司的理由，除了是擴展市場（如近月美國 Groupon Inc.收購香港 uBuyiBuy），另一理由是招攬人才。用幾百萬美元來招聘人才，聽來荒謬，但是一個具執行能力、能夠兼顧市場、營運及技術的團隊得來不易，和普通人員招聘不可相提並論。

又一個可能性是，公司已可獨立營運，以自己的收入擴張。最後一個可能性是上市，但上市需時較長，例如 Facebook，現在的市值已超過四百億美元，但還未公開上市。

(刊於 2010-12-25 香港蘋果日報)

矽巷啁啾：矽谷的融資環境

2011-01-01T00:25:00.003+08:00

一般人很多時會以為科技公司就是簡簡單單搞一個網站，寫一個 iPhone app，多人用就可以賺錢。但是縱觀整個科技工業，由八十年代的 Apple、 Microsoft，到九十年代的 Yahoo!、 Amazon，甚至二千年的 Google、 Facebook或 Salesforce等等，無論是企業家還是投資者，其實都已經累積了好幾十年的人才、人脈和經驗。現在的矽谷，無論是創業還是融資，都已經趨向標準化和系統化。

投資者分三類

在矽谷創業的人，通常對於幾時找投資者、找甚麼投資者、拿多少錢及給多少股份等問題，都已經有一個框架。

譬如 3個剛畢業的大學生自行創業，獲得 Y-Combinator之類的「孵化器」垂青，他們可以預料 Y-Combinator會給他們數萬美元的資本，換大約 6%至 7%的股份，也即是數十萬美元的估值。其他的孵化器可能會用多一點錢，換多一點股份，又或者相反。但是不同的孵化器，對團隊的要求、給予的幫助及對新公司的估值，都有不少類似的地方。

剛開始創業的人除了找孵化器之外，還可以找天使投資者。這些人通常是之前創業成功或者是大型的企業，如 Salesforce或 Cisco創始團隊中的成員。

天使投資者和孵化器相似之處，是他們都比較重視團隊的質素，概念和商業模式反而不是太重要。天使投資者通常會給數萬至十數萬美元的資本，換 2%至 10%的股份，也就是說 100百萬至幾百萬美元的估值。相對孵化器來說，找天使投資者的難度和要求會比較高。

繼天使投資之後的，就是風險投資。風險投資所給的條款變化比較大，但同樣有一定的框架。下一次將會談風險投資。

(刊於 2010-12-18 香港蘋果日報)

How to get 64-bit Firefox 4b7 for Mac today

2010-11-13T18:24:00.006+08:00

Firefox 4b7 has been released for quite some time already, but so far you can't find the 64-bit Mac OS X version in any obvious places. And no.. the usual Mac OS X version you get from Mozilla is a 32-bit binary.

But there's a way to get it...

First, you need to download an older beta, which is publicly availble in 64-bit binary:
ftp://ftp.mozilla.org/pub/firefox/releases/4.0b6/mac64/en-US/Firefox%204.0%20Beta%206.dmg

Install it.

Then, you update it. Help -> Check for Updates...

It will magically update itself to 4.0b7, 64-bit. The update logic in Firefox goes to a weird server called aus2.mozilla.org which doesn't give you any trivial public file listing - so you have to rely on the update process to help you there.

Edit: From FF4b8 and up, the Mac .dmg is already a 64-bit binary by default.

How much faster is your iPhone's CPU compared with your computer?

2010-10-23T07:15:00.007+08:00

Had this strange idea while working in SSE Labs office today.. it's by no means accurate or representative of anything other than OpenSSL performance - it doesn't even use multiple cores - but it gives you some idea of the magnitudes we're talking about.

The test is done by the command
$ openssl speed rsa

It's very easy. Every UNIX computer with OpenSSL can do it.

iPhone 3GS:

Martin-Kous-iPhone:~ root# openssl speed rsa
To get the most accurate results, try to run this
program when this computer is idle.
Doing 512 bit private rsa's for 10s: 1997 512 bit private RSA's in 9.78s
Doing 512 bit public rsa's for 10s: 22561 512 bit public RSA's in 9.80s
Doing 1024 bit private rsa's for 10s: 378 1024 bit private RSA's in 9.81s
Doing 1024 bit public rsa's for 10s: 7353 1024 bit public RSA's in 9.54s
Doing 2048 bit private rsa's for 10s: 62 2048 bit private RSA's in 9.83s
Doing 2048 bit public rsa's for 10s: 2200 2048 bit public RSA's in 9.81s
Doing 4096 bit private rsa's for 10s: 10 4096 bit private RSA's in 10.68s
Doing 4096 bit public rsa's for 10s: 617 4096 bit public RSA's in 9.75s
OpenSSL 0.9.8k 25 Mar 2009
built on: date not available
options:bn(64,32) md2(int) rc4(ptr,char) des(idx,cisc,16,long) aes(partial) idea(int) blowfish(ptr)
compiler: arm-apple-darwin9-gcc -fPIC -fno-common -DOPENSSL_PIC -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -D__DARWIN_UNIX03 -O3 -fomit-frame-pointer -fno-common
available timing options: TIMEB USE_TOD HZ=100 [sysconf value]
timing function used: getrusage
                sign    verify    sign/s verify/s
rsa  512 bits 0.004897s 0.000434s    204.2   2302.1
rsa 1024 bits 0.025952s 0.001297s     38.5    770.8
rsa 2048 bits 0.158548s 0.004459s      6.3    224.3
rsa 4096 bits 1.068000s 0.015802s      0.9     63.3

2007 MacBook Pro:

Martin-Kous-MacBook-Pro:Downloads martinkou$ openssl speed rsa
Doing 512 bit private rsa's for 10s: 25711 512 bit private RSA's in 9.92s
Doing 512 bit public rsa's for 10s: 316964 512 bit public RSA's in 9.93s
Doing 1024 bit private rsa's for 10s: 5190 1024 bit private RSA's in 9.90s
Doing 1024 bit public rsa's for 10s: 93072 1024 bit public RSA's in 8.72s
Doing 2048 bit private rsa's for 10s: 833 2048 bit private RSA's in 9.58s
Doing 2048 bit public rsa's for 10s: 30686 2048 bit public RSA's in 9.75s
Doing 4096 bit private rsa's for 10s: 133 4096 bit private RSA's in 9.88s
Doing 4096 bit public rsa's for 10s: 9285 4096 bit public RSA's in 9.91s
OpenSSL 1.0.0a 1 Jun 2010
built on: Sat Oct  2 20:39:58 PDT 2010
options:bn(64,64) rc4(ptr,char) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: /usr/bin/gcc-4.2 -fPIC -fno-common -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -arch x86_64 -O3 -DL_ENDIAN -DMD32_REG_T=int -Wall
                 sign    verify    sign/s verify/s
rsa  512 bits 0.000386s 0.000031s   2591.8  31919.8
rsa 1024 bits 0.001908s 0.000094s    524.2  10673.4
rsa 2048 bits 0.011501s 0.000318s     87.0   3147.3
rsa 4096 bits 0.074286s 0.001067s     13.5    936.9

Xeon X3450 Server:

martinkou@hydrogen:~$ openssl speed rsa
Doing 512 bit private rsa's for 10s: 102939 512 bit private RSA's in 10.00s
Doing 512 bit public rsa's for 10s: 1143301 512 bit public RSA's in 10.00s
Doing 1024 bit private rsa's for 10s: 21075 1024 bit private RSA's in 10.00s
Doing 1024 bit public rsa's for 10s: 398744 1024 bit public RSA's in 10.00s
Doing 2048 bit private rsa's for 10s: 3418 2048 bit private RSA's in 10.01s
Doing 2048 bit public rsa's for 10s: 115004 2048 bit public RSA's in 10.00s
Doing 4096 bit private rsa's for 10s: 487 4096 bit private RSA's in 10.02s
Doing 4096 bit public rsa's for 10s: 30813 4096 bit public RSA's in 10.00s
OpenSSL 0.9.8k 25 Mar 2009
built on: Thu Aug 12 13:29:53 UTC 2010
options:bn(64,64) md2(int) rc4(ptr,char) des(idx,cisc,16,int) aes(partial) blowfish(ptr2)
compiler: cc -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -m64 -DL_ENDIAN -DTERMIO -O3 -Wa,--noexecstack -g -Wall -DMD32_REG_T=int -DOPENSSL_BN_ASM_MONT -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM
available timing options: TIMES TIMEB HZ=100 [sysconf value]
timing function used: times
                 sign    verify    sign/s verify/s
rsa  512 bits 0.000097s 0.000009s  10293.9 114330.1
rsa 1024 bits 0.000474s 0.000025s   2107.5  39874.4
rsa 2048 bits 0.002929s 0.000087s    341.5  11500.4
rsa 4096 bits 0.020575s 0.000325s     48.6   3081.3

So the average iPhone is roughly 10 to 40x slower than your desktop computer, if we don't include the 1 or 11 other CPU cores in your desktop. If that is included as well.. your desktop computer is easily 100x faster than your iPhone.

Really short way to generate a secure password in Python

2010-09-01T02:36:00.005+08:00

Needed something like that today, was going to write a character table and mix it up with a pseudo-random number generator (i.e. the normal approach), but a thought struck me...

# This assumes you've imported os somewhere in your code.
new_password = os.urandom(8).encode("base64")[0:-2]

Done! QED! FIN!

The only minor problem: It doesn't generate pronounceable passwords, but the passwords I use aren't pronounceable anyways.

Foursquare considered harmful - DON'T USE [Updated w/ 4sq's reply]

2010-08-22T03:07:00.015+08:00

Update [2010-08-31]: foursquare has released their v1.9.2 update on App Store today which should fix the problem.

Update: I posted this on Aug 21st morning and foursquare replied to my blog post with a statement on their HTTPS authentication system rollout that night, so they're working on fixing the issue right now. That's actually a very awesomely fast response. Looking forward to their next update.

Yet Another Update: It has come to my attention from reader fdqps that Gowalla for iPhone is having the same problem. I've run the same test on Gowalla, and unfortunately, I can confirm they're having the same problem as well. So, stop also using Gowalla until they've rolled out an update for this problem.

One More Update: Gowalla has responded this morning that they're rolling out a client update in the near future that'll fix the security problem. Again, very fast response.

Original blog post below:

If you care to intercept the messages your Foursquare app sends out from your iPhone.. you'll see something very unpleasant.

The above screenshot was taken from Wireshark listening in on a gateway Wifi NIC to my iPhone. Take a good look at the "Authorization: Basic" line in the above screenshot - Foursquare sends my account's username and password in plaintext over HTTP, without any encryption. They send it every time you open that Foursquare app. If anybody has access to any routers between you and foursquare, or foursquare's DNS happens to be hijacked by anybody anywhere up your DNS chain.. or someone is setting up a public wifi intercepting foursquare HTTP requests, and you joined it... you're screwed.

Yes, I said they send it every time.

I wasn't aware of the problem until I saw the Gaming foursquare with 9 lines of Perl article from Slashdot this morning. Plenty of people are trying to shorten that 9 lines of Perl on Slashdot.. but the big problem I noticed from the source code was... Why the hell can I send my login/password in plaintext and login to foursquare, at all?!!

So, in conclusion, don't use foursquare until they fixed their security hole. If it's on your mobile phone, remove it right now. Facebook Places is at least decent enough to do the login part via HTTPS.

Update: I see quite some people on Hacker News are having a hard time believing it. You can reproduce this screen with your own Wireshark and any Wifi access point. I used MacBook Pro's Internet Sharing to do it. Here's how you can reproduce it:

Plug your MBP to wired ethernet - you have one of these in your home, right?
Open an ad-hoc network in your MBP, and share Internet access via that ad-hoc network.
Connect your iPhone to your MBP's ad-hoc network, now it should be able to access the Internet via your MBP.
Install and fire up Wireshark in your MBP - make sure to read the instructions or otherwise it can't capture packets.
Get Wireshark to capture packets on the ad-hoc Wifi interface.
Filter out the HTTP packets going to *.foursquare.com by adding the filter 'http.host contains "foursquare"'
Fire up Foursquare on your iPhone, login if you haven't already
Boom, you've just sent your password in plain text, over the Internet.

Update 2: A commenter, kidsat, on Hacker News found Foursquare for Android is doing the same thing. So if it's on your Android phone, remove it too - wait for them to fix the security bug before you go back to use it

List of hidden commands in UNIXKCD

2010-04-01T23:20:00.006+08:00

Shouldn't be complete, I have other things to do today anyways... but here it goes:

pwd

lpr

hello joshua

xyzzy

date

hello

xkcd

fuck

whoami

nano

top

moo

ping

find

your gay

echo

bash

ssh

uname

finger

emacs

vim

asl

goto

find kitten

buy stuff

:(){ :|:& };: (for those who haven't been into the dark side of programming, this is a classical fork bomb)

apt-get

irc

curl

latest

sudo

shutdown

poweroff

logout

cheat

clear

dir

exit

goto

halp

help

light

locate

look

man

poweroff

quit

reboot

restart

make me a sandwich

make love

go [west | east | north | ...]

i read the source code

Many of the commands have variations. Like, you can ask sudo to do different things.

Me on the TV

2009-12-08T00:57:00.004+08:00

It's actually harder to do well in TV interviews than I had expected - definitely harder than newspaper or magazine interviews. The thing is... the reporter would interview you for 2 hours and only take 1 to 3 minutes of it for broadcast. So, if you want to do well, you would have to make sure every single word you utter out in that 2 hours has some impact - and that's really difficult. And on top of that, you need to pronounce every word really clearly - more clear than what you do in everyday conversations - to get a good effect. It's really not easy. Now I'm having a lot more respect for those people who need to appear on TV regularly.

So this is 9th month since Puri! was out - I'm not filthy stinking rich yet, but I've made a good impact already. At least I've transformed myself from a nobody to a very minor somebody :-)

A lot of Hong Kongers still hold a very pessimistic or even a condescending attitude on the tech industry on the whole. So I fully expect a lot of Financial Report/Money Report viewers are looking down on this Martin Kou character as some kind of high tech beggar. Public opinion on the tech industry won't be changing significantly for a while - but, we'll see.

TVB Pearl Money Magazine, 2009-11-27

TVB Jade Financial Magazine, 2009-10-25

The really interesting part of Google Wave - Operational Transform

2009-09-26T03:28:00.006+08:00

Many people wowed at how Google Wave could display changes by multiple clients in real time. Displaying changes alone, however, is rather trivial - you just need to learn DOM ranges, find some way to make a DOM range adapter on IE (IE sucks, in 2009 it doesn't support DOM ranges you need to write an adapter to emulate that out of TextRange), and off you go.

The real juicy part of Google Wave is how it manages to synchronize multiple simultaneous edits correctly, without using any sort of locking mechanisms. When multiple users are editing the same document on Google Wave at the same time, and users updates and commits come at different times due to network latency - users don't get the SVN-like "you've got a conflict here" message or a Trac-like "the document has been modified since the last update" message. Google Wave doesn't tell its users off to resolve the edit conflicts themselves, while still maintaining correct synchronization between edits. That's the impressive part of Google Wave.

Note that, however, by saying "correct synchronization", I'm only meaning that the users' documents on screen would converge to the same thing in a steady state. It doesn't mean it would always converge to a document that makes sense. What OT means to a lay person is that instead of requiring the human to resolve edit conflicts, the computer would try its best to do it.

So, the really impressive part of Google Wave is actually its conflict resolution mechanism. If we draw out a "time line" of conflict resolution mechanisms used in collaborative software, we can have something like this:

Simple locking - if someone is editing a document, you can't edit it. One example is vim on a multi-user UNIX computer.

This is not even feasible on the web when you take network latency into account.
Optimistic locking - you can always edit a document. But if there's any conflict, the system will tell you to resolve it yourself.

The is the most common way of handling edit conflicts on the Internet. MediaWiki, Trac, Bugzilla, SVN, etc. use it. It's usually a good enough solution and is really simple to implement - the simplest way is attaching a revision number.
Operational transformation - you can always edit a document. If there's any conflict, the system resolves it for you and updates you automatically on what it has done.

This is what Google Wave and Etherpad are doing. Google Wave's version should be much more difficult (or, tedious) because its transform algorithm is running on a tree (i.e. XML) with semantics attached to nodes (because the XML is to be rendered to a human readable document). I'm not sure if Google's engineers have come up with some ingenious idea to make the "semantics attached to nodes" part immaterial to their transform algorithms.. but my personal guess is that the tree + semantics part is making their transform algorithms quite a few times more tedious than simply transforming linear text.

So what has Google done in their magical operational transformation whizzbang? Google has a white paper about it, but as Google always do - they only give you a glancing overview of it. The details are in their code.

http://www.waveprotocol.org/whitepapers/operational-transform

CKEditor 3.0 released!

2009-08-23T16:40:00.001+08:00

http://www.ckeditor.com/

Me on the newspaper

2009-08-23T01:24:00.003+08:00

If you've not seen me, I'm the guy in the white T-shirt.

"Cookies need love... like everything else does"

FCKeditor is looking for 2 experienced JavaScript developers in Hong Kong and China

2008-12-07T15:05:00.008+08:00

I've posted this to quite a few places so far, but got no responses. Most probably due to the rather specific and high requirements. So, here's the job, we have two vacancies for people in Hong Kong or greater China:

Company	FCKeditor
Job Title	Advanced JavaScript Developer
Type	Full Time, Remote Work
Requirements	Basic requirements: Experience in JavaScript programming (e.g. knowledge of object oriented and functional programming styles in JavaScript); In-depth knowledge of basic web standards, like XHTML, CSS and DOM, including their intrinsic implementation differences among browsers; Fluent in spoken and written English. Good to have (but not strictly required): Server side programming experience with Java and Ruby on Rails; Experience with FCKeditor; A Bachelor's or Master's degree in Computer Science (or related subjects); Knowledge of open source, and eventually love it :)
Benefits	Work at home - no need to spend time commuting each day; 5-day work week and paid annual leaves; Can enjoy Hong Kong (or China) general public holidays; Excellent exposure to the international information technology industry.
Salary	Depends on ability and experience. 20k+ HKD per month for experienced JavaScript programmers.

So there you have it. It's a really good opportunity for the enterprising geeks to break out of the local tech job market.

If you're interested in the job, please send an email with your CV to jobs@fckeditor.net and cc a copy to martin@fckeditor.net. If you'd like to talk with me before applying, my Skype id is martin_kou. If you're currently studying in or working in HKUST, I can even come to meet you in person.

The irony of it all

2008-10-03T03:17:00.003+08:00

Not only do computers screw up trying to understand human language - whoever wrote this transcript to the lecture also screwed up a little bit... or maybe he's too much into graph theory.

Yet another boost to JavaScript execution performance - Squirrelfish Extreme

2008-09-19T21:45:00.003+08:00

In the past month we've had Mozilla's Tracemonkey and Google Chrome's V8. Today the WebKit team joined the fight with Squirrelfish Extreme, and this fish packs quite a punch.

According to the benchmarks done by Charles Ying, Squirrelfish Extreme is currently the fastest JavaScript engine out there - 54% faster than Tracemonkey and 36% faster than V8 in the SunSpider benchmark.

Not a single word about fast JavaScript engines was mentioned on IEBlog. It seems IE8 is going to hold back the world of web applications for another few years, unfortunately.

BarCampHK 2008

2008-09-10T01:35:00.000+08:00

Images from laihiu's Flickr

Amazing.

It was my second time to a tech conference and the first time I spoke in a session, and quite a few people actually came in and listened to me. Seeing how well the others were doing in other sessions, my presentation skills could definitely use some improvement. Still, I was really happy to see a few audience members have already tried FCKeditor in Drupal and found it to be stable.

I also met quite a few old friends in the conference - my former FYP advisor Dr. Dekai Wu; Yusuf from Outblaze (I had a really hard time telling he was talking about Tracemonkey/Firefox3.1 and FCKeditor to me though, apparently I had a buzzword overflow problem in my brain); Edwin Chu (from secondary school) who's founding a startup right now; Eric Tang (also from my secondary school) who founded HKSUA. It seems my secondary school, QES has been making quite a few tech entrepreneurs in these years, and one of them (Dr. Alan Tam) is already quite successful. Did that entrepreneurship spirit form because... we didn't hand in our homeworks? :P

The other sessions I attended were:

Design UI from Anobii
Anobii's speaker advocated that UI design should be started from hand-coding the HTML, and discouraged the audience from starting web UI design from Photoshop or Dreamweaver's WYSIWYG interface. Dreamweaver's WYSIWYG part sucks of course. But Photoshop? His reason was... not everything you can realize in Photoshop can be realized in HTML/JS/CSS as well. So, if you have a graphics designer working the UI from Photoshop first and then hand it to the web developers, you'll inevitably need a lot of "reconciling" between the web developers and the graphics designers in the form of meetings, arguments, infightings, smashed keyboards... and thus, wasted time.

I have a counter argument to his point, though. No matter how you put it, it is still much easier to create and visualize a beautiful UI layout in Photoshop than in HTML/CSS. e.g. it is much easier to create and adjust a drop shadow, or a border around a round button, in Photoshop. So, I'd wager that there's good possibility that someone who started from hand coded HTML/CSS would need extra time to fine tune his layout after he coded his initial revision. A Photoshopper who already has experience in HTML/JS/CSS, however, would be able to draw and fine tune his layout easily, while making sure that it can be implemented.

The difficulty in my scenario, would be finding the Photoshopper who's skilled in HTML/JS/CSS (or, just HTML/CSS) as well. But you don't really need a Photoshop expert to draw up a good looking UI.
Technology Yesterday "Relevance"
Images from 黛's Flickr

A surprisingly good presentation from a psychologist. The speaker didn't spend much time talking about old technology actually. The whole session was more like a discussion forum on information technology's effect on the society, and its effects to the younger generations (i.e. Gen-Y-ers and the generation after Gen-Y-ers) who grew up with computers.

I can imagine the recent victory of LSD (League of Social Democrats) would be a hot topic in such a forum - had it been hosted two days later - seeing the LSD is currently being supported by a lot of young Internet users. The victory of the LSD is a formal announcement to the importance of the Internet in Hong Kong's political arena - most of the promotions for the LSD were done on the Internet, via Yuk Man's YouTube videos or MyRadio.com.hk. Most of the time Yuk Man didn't even have to take part in such promotions - he spoke something interesting somewhere, one of his audience members recorded down a video of him and uploaded that to YouTube. And, bang! A lot of people saw Yuk Man's speech and rebuttals and were blown away by his impressive oratory skills. A textbook example of how viral marketing works.

A lot of interesting Internet-based promotions in the next Legco election (2012) can be expected, I think. Those who refuse to face reality and adapt, will lose the election game, sooner or later.
Converging Desktop and Web Application
This is again, a discussion forum. Topics included the usual suspects - Google Chrome, WorkerPool API in Google Gears, JIT JavaScript engines, client side storage via DOM Storage API or SQLite, Flash and Silverlight, etc. I was quite surprised by the technical know-how of the audience there. But other than that, no surprises.
Open Source + Microsoft

The topic sounded interesting but I sat so far back in the conference room I couldn't tell what Microsoft's speaker was talking about most of the time. I could hear his point about why Microsoft is entering the open source scene though - there's money to be earned in open source software, entering the scene is just good business practice. There's no conspiracy, and thus open sourcers shouldn't be suspicious of Microsoft's motives.

Well, but that's not gonna change my attitude towards Microsoft. How about fixing IE's standards support to begin with - not just XHTML and CSS, but also ECMAScript (aka JavaScript, e.g. where is Array::lastIndexOf() in IE?!), DOM Selection, DOM Range, XPath, DOM Parser, <canvas>, SVG, MathML, etc. Client side programming in IE sucks so much right now it's like programming with one hand tied to the back of your chair while your other hand has two fingers chopped off. I don't really hate everything that is closed source you know... I don't hate Opera for example. The thing that ticks me off with Microsoft isn't closed source, but incompetence.
Drupal
I didn't attend the whole session because I spent some time talking to Dekai about MathML and FCKeditor after my session. The presenter is a really friendly guy though, and they founded a local Drupal user group right after the session.

Need to sleep now. I'll talk about my session tomorrow or this weekend.

My opinion piece on web browsers in PC Market HK

2008-09-10T01:30:00.003+08:00

The original piece I submitted bashed IE a lot. But the IE bashing was edited out, probably because it was too sensitive for a mainstream computer magazine. The final result looks pretty ok, still.

Real time image editing in JavaScript, with Firefox 3.1

2008-08-24T04:49:00.012+08:00

"JavaScript? That little glue language written by Netscape to try to glue HTML with Java? What a pathetic excuse of a language! It failed its original mission, couldn't deliver 5% of the features of Java, nor 5% of the performance of Java! It's a semi-Java, a quasi-Java, it's the instant coffee of Java, the decaf version of Java, 1mg only, not enough caffeine."

Young JavaScript programmer playing Scott Evil, "But look, Dad! I'm running a freakin' real time image editing application in a freakin' web browser! With JavaScript!"

Look! Really!

p.s. I know full well that describing JavaScript as a light weight version of Java is completely wrong.

p.p.s. Here's the demo author's blog entry about the demo, with source code. And yes, the demo DID calculate the colors of the image pixel-by-pixel.

p.p.p.s. The demo runs under Opera 9.5 as well, with similar performance to Firefox 3.0. It doesn't work in Safari 3.1. IE (even IE8 beta) doesn't support canvas and has very bad JavaScript performance in general so there's no chance it can work within a few years. Boo, Microsoft.

Watch out for when your iPhone 3G refuses to receive calls

2008-08-21T09:19:00.006+08:00

I went out to Hang Hau for breakfast today, with my iPhone in my pocket. When I got back home, my mom asked me why I didn't answer her phone calls.

Hmm... maybe it's because I didn't take my headphones with me this morning? The ringtone is quite hard to hear in all the traffic noise you know. But as soon as my mom did a test call to my phone, I knew the problem is with the phone.

The phone never rang.

The phone was turned on and I could run apps and go online via wifi perfectly in it. There're 2 bars of signal from 3's service. But the phone part of it was like.. disabled! I couldn't call it from landline or from other mobile phones, and it couldn't call out. Worse, while the iPhone was on but not accepting any calls, the caller wasn't able to leave voice messages either. For a moment I was suspecting it was 3's problem. But the phone worked correctly after I rebooted it. So there must be some problem with my iPhone that prevented it from receiving calls and calling out.

The mobile phone part of the iPhone can crash silently. That's a big problem. I don't care too much if SFNetNews or Bloomberg crashes - though I would really appreciate it if Apple can stop them from crashing; But why can I miss calls with my phone turned on, connected with cell towers?

I tried to replicate the problem by killing the MobilePhone app in my iPhone and calling in, after the reboot. I could still hear the ringtone and answer the call, however. Something much more complicated than just an app crash must have happened this morning. Not enough memory left for the incoming call?

A taste of what's coming in CKEditor 3.0

2008-08-13T18:39:00.009+08:00

CKEditor 3.0 will be a major rewrite to the very successful FCKeditor 2.6.x series. Currently it is still in the prototypal stage (read: most of the features are not done) but it's already showing some amazing functionalities.

e.g. This (sorry about the video artifacts):

You can view the screencast in its full resolution here.

It is not impossible with FCKeditor 2.6.x right now or other RTE libraries out there at the moment. But very little JavaScript trickery is needed to make it work with CKEditor 3.0, and the switching speed is very fast. Imagine a Wiki with that.

Things the iPhone 3G technical specs don't tell you...

2008-08-13T17:35:00.010+08:00

You can get that by jailbreaking your iPhone and run sysctl -a in the terminal.

Here is what I have on my iPhone 3G. with 2.0.1 firmware:

kern.ostype = Darwin
kern.osrelease = 9.3.1
kern.osrevision = 199506
kern.version = Darwin Kernel Version 9.3.1: Sun Jun 15 21:37:01 PDT 2008; root:xnu-1228.6.76~45/RELEASE_ARM_S5L8900X
kern.maxvnodes = 640
kern.maxproc = 52
kern.maxfiles = 12288
kern.argmax = 262144
kern.securelevel = 0
kern.hostname = iPhone
kern.hostid = 0
kern.clockrate: hz = 100, tick = 10000, profhz = 100, stathz = 100
kern.posix1version = 200112
kern.ngroups = 16
kern.job_control = 1
kern.saved_ids = 1
kern.boottime = Tue Aug 12 20:13:51 2008

kern.nisdomainname =
kern.maxfilesperproc = 10240
kern.maxprocperuid = 26
kern.dummy = 0
kern.dummy = 0
kern.usrstack = 805306368
kern.dummy = 0
kern.dummy = 0
kern.dummy = 0
kern.exec: unknown type returned
kern.aiomax = 10
kern.aioprocmax = 4
kern.aiothreads = 2
kern.corefile = /cores/core.%P
kern.delayterm = 0
kern.shreg_private = 0
kern.proc_low_pri_io = 0
kern.usrstack64 = 8247063986311266304
kern.procname =
kern.speculative_reads_disabled = 0
kern.osversion = 5B108
kern.safeboot = 0
kern.rage_vnode = 0
vfs.hfs has 2 mounted instances
vfs.devfs has 1 mounted instance
hw.machine = iPhone1,2
hw.model = N82AP
hw.ncpu = 1
hw.byteorder = 1234
hw.physmem = 121634816
hw.usermem = 96317440
hw.pagesize = 4096
hw.epoch = 1
hw.vectorunit = 0
hw.busfrequency = 103000000
hw.cpufrequency = 412000000
hw.cachelinesize = 32
hw.l1icachesize = 16384
hw.l1dcachesize = 16384
hw.tbfrequency = 6000000
hw.memsize = 121634816
hw.availcpu = 1
user.cs_path = /usr/bin:/bin:/usr/sbin:/sbin
user.bc_base_max = 99
user.bc_dim_max = 2048
user.bc_scale_max = 99
user.bc_string_max = 1000
user.coll_weights_max = 2
user.expr_nest_max = 32
user.line_max = 2048
user.re_dup_max = 255
user.posix2_version = 200112
user.posix2_c_bind = 0
user.posix2_c_dev = 0
user.posix2_char_term = 0
user.posix2_fort_dev = 0
user.posix2_fort_run = 0
user.posix2_localedef = 0
user.posix2_sw_dev = 0
user.posix2_upe = 0
user.stream_max = 20
user.tzname_max = 255
kern.ostype: Darwin
kern.osrelease: 9.3.1
kern.osrevision: 199506
kern.version: Darwin Kernel Version 9.3.1: Sun Jun 15 21:37:01 PDT 2008; root:xnu-1228.6.76~45/RELEASE_ARM_S5L8900X
kern.maxvnodes: 640
kern.maxproc: 52
kern.maxfiles: 12288
kern.argmax: 262144
kern.securelevel: 0
kern.hostname: iPhone
kern.hostid: 0
kern.clockrate: { hz = 100, tick = 10000, tickadj = -1072645529, profhz = 100, stathz = 100 }
kern.posix1version: 200112
kern.ngroups: 16
kern.job_control: 1
kern.saved_ids: 1
kern.boottime: { sec = 1218543231, usec = 0 } Tue Aug 12 20:13:51 2008
kern.nisdomainname:
kern.maxfilesperproc: 10240
kern.maxprocperuid: 26
kern.ipc.maxsockbuf: 8388608
kern.ipc.sockbuf_waste_factor: 8
kern.ipc.somaxconn: 128
kern.ipc.nmbclusters: 3486
kern.ipc.soqlimitcompat: 1
kern.ipc.mb_normalized: 0
kern.ipc.sosendjcl_ignore_capab: 0
kern.ipc.sosendjcl: 1
kern.ipc.sorecvmincopy: 16384
kern.ipc.sosendminchain: 16384
kern.ipc.soqlencomp: 0
kern.ipc.njclbytes: 0
kern.ipc.njcl: 0
kern.ipc.sbspace_factor: 8
kern.ipc.maxsockets: 128
kern.dummy: 0
kern.usrstack: 805306368
kern.aiomax: 10
kern.aioprocmax: 4
kern.aiothreads: 2
kern.corefile: /cores/core.%P
kern.delayterm: 0
kern.shreg_private: 0
kern.proc_low_pri_io: 0
kern.posix.sem.max: 10000
kern.usrstack64:
kern.tfp.policy: 2
kern.procname:
kern.speculative_reads_disabled: 0
kern.osversion: 5B108
kern.safeboot: 0
kern.lctx.max: 8192
kern.lctx.count: 0
kern.lctx.last: 1
kern.rage_vnode: 0
kern.tty.ptmx_max: 127
kern.sleeptime: { sec = 1218607962, usec = 525806 } Wed Aug 13 14:12:42 2008
kern.waketime: { sec = 1218608306, usec = 11 } Wed Aug 13 14:18:26 2008
kern.hibernatefile:
kern.bootsignature:
kern.hibernatemode: 0
kern.monotonicclock: 1218648807
kern.maxnbuf: 558
kern.nbuf: 558
kern.flush_cache_on_write: 0
kern.always_do_fullfsync: 0
kern.sugid_scripts: 0
kern.affinity_sets_mapping: 1
kern.affinity_sets_enabled: 1
kern.singleuser: 0
kern.bootargs:
kern.memorystatus_kev_failure_count: 0
kern.memorystatus_level: 50
kern.msgbuf: 4096
kern.wq_timer_interval_msecs: 40
kern.wq_max_run_latency_usecs: 500
kern.wq_reduce_pool_window_usecs: 3000000
kern.wq_stalled_window_usecs: 20000
kern.secure_kernel: 1
kern.wakereason: usb
vm.loadavg: { 0.00 0.29 0.39 }
vm.swapusage: total = 0.00M used = 0.00M free = 0.00M
vm.cs_debug: 0
vm.cs_force_hard: 0
vm.cs_force_kill: 0
vm.user_wire_limit: -2302491336873346024
vm.global_user_wire_limit: -4602390231608460264
vm.cs_blob_size_max: 38768
vm.cs_blob_size_peak: 419168
vm.cs_blob_count_peak: 172
vm.cs_blob_size: 411472
vm.cs_blob_count: 164
vm.cs_validation: 1
vm.vm_page_free_target: 374
vm.shared_region_persistence: 1
vm.shared_region_version: 3
vm.shared_region_trace_level: 1
net.local.stream.recvspace: 8192
net.local.stream.sendspace: 8192
net.local.dgram.recvspace: 4096
net.local.dgram.maxdgram: 2048
net.local.inflight: 0
net.inet.ip.portrange.hilast: 65535
net.inet.ip.portrange.hifirst: 49152
net.inet.ip.portrange.last: 65535
net.inet.ip.portrange.first: 49152
net.inet.ip.portrange.lowlast: 600
net.inet.ip.portrange.lowfirst: 1023
net.inet.ip.forwarding: 0
net.inet.ip.redirect: 1
net.inet.ip.ttl: 64
net.inet.ip.rtexpire: 2400
net.inet.ip.rtminexpire: 10
net.inet.ip.rtmaxcache: 128
net.inet.ip.sourceroute: 0
net.inet.ip.intr_queue_maxlen: 50
net.inet.ip.intr_queue_drops: 0
net.inet.ip.accept_sourceroute: 0
net.inet.ip.fastforwarding: 0
net.inet.ip.keepfaith: 0
net.inet.ip.subnets_are_local: 0
net.inet.ip.use_route_genid: 1
net.inet.ip.check_route_selfref: 1
net.inet.ip.linklocal.in.allowbadttl: 1
net.inet.ip.check_interface: 0
net.inet.ip.maxfrags: 216
net.inet.ip.maxfragsperpacket: 128
net.inet.ip.maxfragpackets: 108
net.inet.ip.maxchainsent: 8
net.inet.icmp.maskrepl: 0
net.inet.icmp.icmplim: 250
net.inet.icmp.timestamp: 0
net.inet.icmp.bmcastecho: 1
net.inet.icmp.log_redirect: 0
net.inet.icmp.drop_redirect: 0
net.inet.tcp.rfc1323: 1
net.inet.tcp.rfc1644: 0
net.inet.tcp.mssdflt: 512
net.inet.tcp.keepidle: 7200000
net.inet.tcp.keepintvl: 75000
net.inet.tcp.sendspace: 131072
net.inet.tcp.recvspace: 131072
net.inet.tcp.keepinit: 75000
net.inet.tcp.rexmt_thresh: 2
net.inet.tcp.rfc3465: 1
net.inet.tcp.maxseg_unacked: 8
net.inet.tcp.slowlink_wsize: 8192
net.inet.tcp.reass.overflows: 0
net.inet.tcp.reass.cursegments: 0
net.inet.tcp.reass.maxsegments: 217
net.inet.tcp.drop_synfin: 1
net.inet.tcp.tcp_lq_overflow: 1
net.inet.tcp.delayed_ack: 3
net.inet.tcp.blackhole: 0
net.inet.tcp.log_in_vain: 0
net.inet.tcp.socket_unlocked_on_output: 1
net.inet.tcp.packetchain: 50
net.inet.tcp.ecn_negotiate_in: 0
net.inet.tcp.ecn_initiate_out: 0
net.inet.tcp.newreno: 0
net.inet.tcp.local_slowstart_flightsize: 8
net.inet.tcp.slowstart_flightsize: 1
net.inet.tcp.path_mtu_discovery: 1
net.inet.tcp.sack_globalholes: 0
net.inet.tcp.sack_globalmaxholes: 65536
net.inet.tcp.sack_maxholes: 128
net.inet.tcp.sack: 1
net.inet.tcp.rtt_min: 1
net.inet.tcp.background_io_enabled: 1
net.inet.tcp.isn_reseed_interval: 0
net.inet.tcp.strict_rfc1948: 0
net.inet.tcp.icmp_may_rst: 1
net.inet.tcp.pcbcount: 8
net.inet.tcp.do_tcpdrain: 0
net.inet.tcp.tcbhashsize: 128
net.inet.tcp.minmssoverload: 0
net.inet.tcp.minmss: 216
net.inet.tcp.always_keepalive: 0
net.inet.tcp.msl: 15000
net.inet.tcp.background_io_trigger: 5
net.inet.tcp.sockthreshold: 0
net.inet.tcp.out_sw_cksum_bytes: 2439757
net.inet.tcp.out_sw_cksum: 32018
net.inet.tcp.in_sw_cksum_bytes: 46445398
net.inet.tcp.in_sw_cksum: 41658
net.inet.tcp.win_scale_factor: 3
net.inet.udp.checksum: 1
net.inet.udp.maxdgram: 9216
net.inet.udp.recvspace: 41600
net.inet.udp.pcbcount: 5
net.inet.udp.blackhole: 0
net.inet.udp.log_in_vain: 0
net.inet.udp.out_sw_cksum_bytes: 264471
net.inet.udp.out_sw_cksum: 3750
net.inet.udp.in_sw_cksum_bytes: 376819
net.inet.udp.in_sw_cksum: 3501
net.inet.ipsec.def_policy: 1
net.inet.ipsec.esp_trans_deflev: 1
net.inet.ipsec.esp_net_deflev: 1
net.inet.ipsec.ah_trans_deflev: 1
net.inet.ipsec.ah_net_deflev: 1
net.inet.ipsec.ah_cleartos: 1
net.inet.ipsec.ah_offsetmask: 0
net.inet.ipsec.dfbit: 0
net.inet.ipsec.ecn: 0
net.inet.ipsec.debug: 0
net.inet.ipsec.esp_randpad: -1
net.inet.ipsec.esp_port: 0
net.inet.ipsec.bypass: 1
net.inet.raw.recvspace: 8192
net.inet.raw.maxdgram: 8192
net.link.ether.inet.send_conflicting_probes: 1
net.link.ether.inet.keep_announcements: 1
net.link.ether.inet.log_arp_warnings: 0
net.link.ether.inet.sendllconflict: 0
net.link.ether.inet.proxyall: 0
net.link.ether.inet.useloopback: 1
net.link.ether.inet.maxtries: 5
net.link.ether.inet.apple_hwcksum_rx: 1
net.link.ether.inet.apple_hwcksum_tx: 1
net.link.ether.inet.host_down_time: 20
net.link.ether.inet.max_age: 1200
net.link.ether.inet.prune_intvl: 300
net.key.debug: 0
net.key.spi_trycnt: 1000
net.key.spi_minval: 256
net.key.spi_maxval: 268435455
net.key.int_random: 60
net.key.larval_lifetime: 30
net.key.blockacq_count: 10
net.key.blockacq_lifetime: 20
net.key.esp_keymin: 256
net.key.esp_auth: 0
net.key.ah_keymin: 128
net.key.prefered_oldsa: 0
net.key.natt_keepalive_interval: 20
debug.wdtlog: 2114u0 2114u0 2100m0 2000m0 1914u0 1914u0 1914u0 1914u0 1900m0 1800m0 1714u0 1714u0 1714u0 1714u0 1700m0 1600m0 1514u0 1514u0 1514u0 1514u0 1500m0 1400m0 1314u0 1314u0 1314u0 1314u0 1300m0 1200m0 1110u0 1110u0 1110u0 1110u0 1100m0 1000m0 909u0 909u0 909u0 909u0 900m0 800m0 709u0 709u0 709u0 709u0 700m0 600m0 507u0 507u0 507u0 507u0 500m0 400m0 307u0 307u0 307u0 307u0 300m0 200m0 107u0 107u0 107u0 107u0 100m0 0m0
debug.marvel_debug: 136 136
debug.lowpri_max_waiting_msecs: 200
debug.lowpri_max_window_msecs: 200
debug.lowpri_IO_window_inc: 50
debug.lowpri_IO_initial_window_msecs: 100
debug.bpf_maxdevices: 256
debug.bpf_maxbufsize: 524288
debug.bpf_bufsize: 4096
debug.iokit: 0
hw.ncpu: 1
hw.byteorder: 1234
hw.memsize: 121634816
hw.activecpu: 1
hw.optional.floatingpoint: 1
hw.packages: 1
hw.tbfrequency: 6000000
hw.fixfrequency: 24000000
hw.prffrequency_max: 51500000
hw.prffrequency_min: 51500000
hw.prffrequency: 51500000
hw.memfrequency_max: 137333333
hw.memfrequency_min: 137333333
hw.memfrequency: 137333333
hw.l1dcachesize: 16384
hw.l1icachesize: 16384
hw.cachelinesize: 32
hw.cpufrequency_max: 412000000
hw.cpufrequency_min: 412000000
hw.cpufrequency: 412000000
hw.busfrequency_max: 103000000
hw.busfrequency_min: 103000000
hw.busfrequency: 103000000
hw.pagesize: 4096
hw.cachesize: 0 0 0 0 0 0 0 0 0 0
hw.cacheconfig: 0 16384 16384 1 32 4 0 0 0 0
hw.cpufamily: -1879695144
hw.cpu64bit_capable: 0
hw.cpusubtype: 6
hw.cputype: 12
hw.logicalcpu_max: 1
hw.logicalcpu: 1
hw.physicalcpu_max: 1
hw.physicalcpu: 1
machdep.alignmenttrap: 0
security.mac.seatbelt.debug: 0
security.mac.seatbelt.profile_refcount: 15
security.mac.seatbelt.qtnstate_refcount: 0
security.mac.seatbelt.cred_label_refcount: 5
security.mac.seatbelt.disable_builtins: 0
security.mac.vnode_enforce: 1
security.mac.vm_enforce: 1
security.mac.sysvshm_enforce: 1
security.mac.sysvsem_enforce: 1
security.mac.sysvmsg_enforce: 1
security.mac.system_enforce: 1
security.mac.socket_enforce: 1
security.mac.proc_enforce: 1
security.mac.posixshm_enforce: 1
security.mac.posixsem_enforce: 1
security.mac.pipe_enforce: 1
security.mac.iokit_enforce: 0
security.mac.file_enforce: 0
security.mac.device_enforce: 1
security.mac.mmap_revocation_via_cow: 0
security.mac.mmap_revocation: 0
security.mac.max_slots: 8

Interesting bits:

The CPU has a maximum frequency of 412MHz, but I don't know whether it scales down its running frequency in power saving mode or not. Outside info says it's running at 620MHz.
The CPU comes with 32KB of L1 cache, 16KB for data and 16KB for instructions. There's no sign of any L2 cache.
Apple has chosen to run the ARM CPU in little endian mode.
The CPU comes with an FPU as well.
The CPU chip seems to be something called S5L8900(X?) from Samsung. Samsung's datasheet says it's an ARM CPU with built-in PowerVR MBX 3D graphics.
The usable memory size is exactly 116MB. DRAM chips don't come in non-round numbers, the closest guess to the total physical DRAM size would be 128MB. So, 12MB is gone for other uses... graphics?
It doesn't seem to use any swap memory at all. So if the 116MB runs out, your apps would surely crash.
There're a few hw.memfrequency entries in the iPhone's sysctl listing which do not exist in Macintosh computers. It seems to indicate the iPhone's memory is running at roughtly 137MHz, I wonder if there's any DDR trickery involved.
There's a "net.inet.ip.forwarding" entry which is set to 0 by default but it can be set to 1. In Linux that is the first step in making an NAT router.

How to properly fix the Mac OS X ARDagent security hole

2008-06-21T08:19:00.013+08:00

Slashdot posted about a root privilege escalation bug in Mac OS X a few days ago. It made quite a big fuzz among Apple users because Mac OS X, which is more UNIX than Linux; which is created, coded and certified by the infallible bearded UNIX wizards; must be... eh... infallibly secure! This seems to be not the case now, as any would-be OSX virus writer could just use this hole to gain root privileges in their creations without the user noticing.

Many blogs have posted about temporary fixes to the situation, but many of their fixes brings with problems on their own. Here're some of those half-working fixes I've seen:

Remove the setuid bit in the ARDAgent executable.
This approach actually makes a lot of sense if all you need to tackle is the security hole itself. But it breaks the Apple Remote Management service so if you try to remotely control your Mac with Apple's Remote Desktop software (you need to buy it from Apple), it will no longer work.
Start up the Apple Remote Management service.
Amazingly, actually starting up the Apple Remote Management service - the very thing that caused the security hole - seems to close the security hole! Running the dreaded exploit script osascript -e 'tell app "ARDagent" to do shell script "whoami"'after starting up ARM would give you an error message instead of the "root" message. But does this "put out fire with fire" approach make you nervous? What if it goes wrong?

To see what happens if it goes wrong, just restart the ARM service in System Preferences, and try the exploit script again. You've been rooted! So this method does not actually work.

A better fix to the issue would be one that does not break Apple Remote Desktop, while actually preventing Apple Script from rooting your machine at the same time no matter whether the Apple Remote Management service is on or not. And here's how:

Edit the file /System/Library/CoreServices/RemoteManagement/ARDAgent.app/Contents/Info.plist as root.
Add the following two lines just before </dict>:
<key>NSAppleScriptEnabled</key> <string>YES</string>
Save it.
Start and stop (or stop and start) Apple Remote Management service.

Now try the exploit script again, you should get:
23:47: execution error: ARDAgent got an error: "whoami" doesn’t understand the do shell script message. (-1708)

No matter whether ARM is started or not.

More devils in the details

2008-05-17T22:23:00.003+08:00

The lights.

Apparently the owner of the house thought he should live like a rich and so he installed a roof lamp with those artsy lamp shades in the living room. It looks nice but those artsy glassy lamp shades decrease the energy efficiency of the lamp dramatically. The lamp has six light bulbs, each rated at 60W (so it's a f**king 360W!), but the living room still looked dark at night. This must be one of the most ridiculous things I've seen - you use 360W and you still can't light up a <150 sq. ft. living room properly.

So I bought a floor lamp from Hung Hom, with energy-saving fluorescent bulbs. Another $600+ spent. Moving it from Hung Hom to Tai Po Tsai was NOT fun.

The new flat itself, and the little details...

2008-05-17T21:51:00.010+08:00

The new flat is small but nice, really nice, almost too nice for its lowly monthly rent. It is visible from the drive to HKUST, one of those houses in the first row in the photo below.

The flat itself is in the back of the house so I don't get a view to Flower and Food BBQ directly out of the window. But that's simply too much to ask for with the cheap price I'm paying.

They say the devil is in the details, but for such a house I'm quite happy to deal with the details. There are a lot of such "details"...

First, the TV reception is poor.

TVB Jade is the best channel I can get here, and still the picture is not so clear. I've tried to improve the reception by buying a signal amplifier ($500) but that didn't seem to help. The TV antenna socket does not work, I only had a bare antenna cable from the window to work with, and I had to install the plug myself.

The amplifier didn't seem to work, unfortunately. It is now suspected that the antenna cable itself is broken and so I had to buy a new antenna cable from Sham Shui Po today ($900!). I've never imagined 20m of coaxial cable costs that much...

I've also noticed that Wing Shing (one of those electronics shops in SSP Ap Liu Street) lied about the signal amplifier in their receipt - the signal amplifier only has a gain of 21dB, but the receipt says it's 29dB.

The second to fourth days of moving...

2008-05-17T11:52:00.009+08:00

I connected the ADSL modem to the telephone jack in the second day I moved in, but there was simply no signal. So for a few days I had to go to HKUST for Internet access.

While I was back home without Internet access in the few days, I tried to do some experiments with the iPhone SDK from Apple. Since I don't have an iPhone but I want to develop web applications that can be used in iPhone, I'm very interested in the Safari browser inside Apple's iPhone simulator. So I downloaded Apple's "Getting Started" videos along with the iPhone SDK while I had Internet access in HKUST.

A little JavaScript code thrown into the iPhone simulator reveals that it is actually using the Safari browser in my MacBook Pro, so it's not a separate Safari installation. This is a little bit discomforting... Would everything running ok in the simulator's Safari browser run in an actual iPhone as well?

This is where I got my Internet access in the first few days. I was usually sitting in the Library, in the Cafe or in the new wing area of HKUST (in photo). I was actually a little bit worried when I left my MBP alone while getting off for a drink or for the toilet; but then, I've done that for 3 years and my notebook never got stolen. HKUST is an incredibly nice place by the way, I would have studied my Master's here right after the Bachelor's degree if I could, but I have a family to feed. Even scholarships plus a TA job won't be enough for my family's expenditures. Being a poor man in a rich city sucks a lot.

Strangely, in the few days I sat in UST for Internet access, I've only met one familiar person. I thought I could ask some of my previous schoolmates who're taking Master's now for a lunch in those few days.

Things to write about in the coming days.

2008-05-13T17:31:00.005+08:00

I have the memory of a goldfish - I forget things soon after they're spoken to me, which is why I rely on emails and IM/IRC chat logs a lot. So I'd also need a note about what to write here in the days when I have less things to do.

The progress of moving.
New features and important bugfixes in FCKeditor 2.5, 2.6 and the up-and-coming 2.6.1.
Recent developments in the web app field:
1. CSS transitions and animations - no JavaScript needed
2. Client-side storage via SQL in JavaScript
3. JavaScript dynamic linking
4. DOM selectors and their relative performance
5. The astute reader might have noticed some the above topics are related to HTML5
Some highly aggravating IE bugs I met during FCKeditor development:
1. Selections and non-standard-conforming ranges in IE
2. Memory leaks - this is an old topic, but there are more leaks beyond circular references
3. Why you should never use Microsoft-specific functions in IE - the horrible bugs and ridiculous workarounds with window.createPopup().
Startup ideas (??)

First day in the new flat - continued

2008-05-13T11:44:00.004+08:00

Without an internet connection in the new home, I can't really be spraying bugs in FCKeditor. So what's left for me to do (except moving stuff around)? How many bits are needed to perfectly encode every possible key that can be made form the key mold that this key came from?

The first day in the new flat

2008-05-13T11:19:00.004+08:00

10th of May

The first things I moved to the new flat were my computers (there are four of them) and all my computer-related equipments, and a bed roll. That was a hell lot of equipments to move around, and my arms are still aching right now.

The new flat wouldn't have Internet connection until the 15th of May though, so all I could do with the desktops was playing games. The notebooks are much more useful now, because I can bring them to the university and go online via wifi.

There's no computer desk in the new flat, yet. So my computers are all sitting on the floor near my bedroll.

Long time no blog - moving home

2008-05-13T10:30:00.006+08:00

It's been quite a while since the last time I wrote anything here. I've always been wanting to write about new features in FCKeditor and those pesky IE bugs but I never got the time nor the will to actually write a post for that. But now I'm on my long annual leave from the FCKeditor project, and I'm also moving to a new home. So I figured it's a good time to take up blogging again while I have nothing to do.

I've been moving home from To Kwa Wan (a poor, dirty urban district in Hong Kong) to Tai Po Tsai (a suburban village near HKUST) since the 10th of May. I moved to the To Kwa Wan flat this time last year because of the ridiculously cheap rent ($2.8k/mo), and that I wanted to save money for the future. I have to rent private flats because I'm not eligible for public housing or even semi-public housing in Hong Kong, because the government thinks I'm a "middle class"... But come on, how can someone without his own house and his own car be a middle classman?! Stupid policies... There's no wonder why Hong Kong is getting near (or is it above now?) places like Brazil in income inequality and having more than a million living in poverty these days with a government like this one.

Anyway... back to the topic. The old flat in To Kwa Wan was butt ugly. I can't regret my "save money" decision more. For HK$2.8k/month I got a tiny studio flat in a slum area to live in. I thought it was ok as long as the inside of the flat is clean but I was wrong... The nasty shitty smells from the street outside the windows and from the corridor outside the door constantly invaded the place; every corner of the corridor is filled with spiders; the staircases to the outside are often filled with the smell of urine... If you want to experience the lives of the ultra poor in Hong Kong, that is the place you want to go to, and after that you won't be able to comprehend why Hong Kong can be regarded as a first-world place, at all. The only thing worse than that is living under a bridge. Why is the government not giving me public housing... again? I'm not poor enough?

Ok, so I got fed up with that shitty (in a literal way) flat, and decided to move to somewhere clean, with sunlight and fresh air. I still need to save money up so I can't be renting those $10k+/month sea view flats even though I can afford to... So I have to look into the suburban area. And there's just one such village (the Tai Po Tsai village) just outside my previous place of study, how convenient. The new flat is a little bit larger than 300 sq. ft. - not large by any measure, but enough for me and my mom. Right outside the door is a row of tropical trees, a grass field and a BBQ place (Flower and Food BBQ) still further. A huge improvement to the dark corridor filled with sewage and spider webs that was in To Kwa Wan. If there's a smell in the new flat, it's the faint smell of barbecued food, which is quite pleasant actually.

I started moving to the new home at the 10th of May, and as of today the furniture and the broadband connection aren't moved to the new flat yet - so I'm now having to go online with the free wifi offered by HKUST - just next doors. That's still not bad.

New year presents from Opera

2007-12-31T18:14:00.000+08:00

Thank you Hallvord :D

Also, a late thank you to Yusuf from Outblaze Ltd. for the FON router and T-shirt (but most importantly, the experience sharing).

GWT's solution to the JavaScript dynamic memory leak problem.

2007-12-27T14:43:00.000+08:00

Any experienced AJAX developer should have seen this: You wrote a web application with a lot of JavaScript logic for the front end. It looks beautiful, it runs fast, it passes all feature tests. But there's one problem: it occupies more and more memory as it runs.

There's no way you can prevent it in Firefox, due to memory fragmentation at the very least (technically that is not memory leak, but that makes no difference to users), caches, and possible memory mismanagement by plugins. But memory leaks in IE are preventable, and it was one of the major topics for my FYP - WT Toolkit. The idea is trying to solve the memory leak problem as far as possible, and leave it to the browser developers (Firefox/Gecko, that is) if it isn't possible to solve in JavaScript.

WT Toolkit's approach is straightforward - since memory leaks in IE are usually made from circular references between DOM nodes and JavaScript objects (including functions), we try to eliminate the need to reference to any DOM nodes in JavaScript programming. It's simple in theory, but difficult to implement in practice.

Now I've graduated from college, but I'm facing the same problem in another project - FCKeditor. More precisely, I'm facing the problem in FCKeditor's floating dialog branch. Tools like Drip or sIEve, while excellent, can only help a bit in detecting JavaScript memory leaks - it is entirely possible that both of them do not detect any memory leaks but your application still contains dynamic leaks (i.e. increasing memory usage BEFORE page refresh), and I've talked about that in my FYP report.

So now I'm having to do the same research all over again. But this time the problem is harder - for WT Toolkit, I could simply look over the existing literature about the nature of the IE memory leaks, build the toolkit in a way that it can be mathematically proved that it cannot leak under certain circumstances (the "circumstances" here have to be flexible enough to make the toolkit practical, of course), and only develop web applications under the allowed circumstances. FCKeditor, on the other hand, does not have a toolkit built with such provisions.

Right... So I have to read the literature on JavaScript memory leaks all over again, and code again, how wonderful. While reading the literatures I found a piece of gem on this topic from the Google Web Toolkit team: http://code.google.com/p/google-web-toolkit/wiki/DomEventsAndMemoryLeaks.

It discusses Google's solution to the very same problem that I tried to solve in WT Toolkit. It's not ideal, but it's simpler than WT Toolkit's approach. The only thing that it assumes is that the web developer won't call element.removeChild(widget) and won't make his own reference cycles in JavaScript - not a big problem for GWT since the web developer is supposed to write in Java.

Got Mac OS X Leopard today.

2007-12-22T21:31:00.000+08:00

It got me thinking... How many screw ups does it take to end up with Windows Vista's current UI design, given they already had a good example to copy from?

WriteArea: Use FCKeditor everywhere you want!

2007-12-08T04:57:00.000+08:00

How many times you have encountered this: You want to write a blog post - it may be on Blogger, it may be on Xanga, it may be on Wordpress - but the HTML editor provided by the blog site does not do what you want. Say, you want to draw a table with Xanga's HTML editor, which button is that? Well, sorry, there's no such thing. How about making indented list items in Blogger? No can do. Wanting to add a Flash animation in your blog post to spice things up a bit? Well, you need to resort to editing HTML source - a tedious, painful, and error prone process for non-professionals.

Introducing WriteArea

WriteArea is a Firefox plugin that lets you write your blog posts, or anything requiring the use of HTML, with FCKeditor - the most popular web based rich text editor of the world. It is a new Firefox plugin that lets you convert any HTML text areas in any web site into a full-featured FCKeditor editing dialog. With it, you can create tables in your Xanga posts, you can create indented list items in Blogger, you can add Flash animations to your blog posts without fiddling with arcane HTML code, you can have the ability to write things in _subscript and ^superscript, and the ability to edit the properties of tables and images with right click menues... WriteArea is a little plugin gives you a whole world of possibilities in web authoring.

Using WriteArea

WriteArea can be activated from any text area boxes of any website. For blog sites, that is usually the text area provided by the "Edit Source" or "Edit HTML" feature. To give you a rough idea of how WriteArea works, I'll give you an example how I wrote this post with WriteArea in Blogger.

Step 1: Switch to "Edit HTML" in Blogger, and activate the plugin by right clicking on the text area.

Step 2: Write your message in the WriteArea dialog that pops up.

Step 3: Click "Save" in the WriteArea dialog to save the HTML code into the blog site's text area. If you want to edit your post again just right click on the text area and activate WriteArea again.

Step 4: Publish your blog post when you've done editing it.

Getting WriteArea

WriteArea can be downloaded here. Since the plugin is new, it is being sandboxed right now and is available only to registered users in http://addons.mozilla.org. You'll also need to enable showing sandboxed plugins in your user preferences after you've registered and logged into the Mozilla plugin site before downloading it. A plugin being sandboxed also means it is a beta and thus it's likely it will have some bugs. So, be a responsible open source software user, report any bugs you've found and write a favorable review for it if you think it is useful.

Early history of the infotech industry - Triumph of the Nerds

2007-09-18T00:51:00.001+08:00

From the first affordable microcomputer (MITS Altair 8800), to the Homebrew Computer Club in Stanford University, to IBM and Microsoft's deals and struggles, to Netscape and the beginning of the web just before the 21st century. I almost forgot the documentary's name when someone asked me about the history between IBM and Microsoft today. So I figured, I should write this down.

It is not a computer science lecture, but interestingly, it tells me more about how far the infotech industry has come - CPUs, compilers, operating systems, networking, 3D graphics, artificial intelligence; accounting packages, customer relationship management systems (CRM), enterprise resource planning (ERP), electronic communication network (ECN), quant funds, Google... it all started from people playing with boxes without keyboards, monitors or even printers. The substance behind the tech IPO crazies, tech bubbles, and Web 2.0, is not just hard work. Simply working hard cannot do these.

"Do you want to spend the rest of your life selling sugared water, or do you want a chance to change the world?"

And Apple's 1984 advertisement... it may look like just another annoying advertisement in the eyes of the public. But for anyone who's read George Orwell, he knows whoever behind the advertisement, has a far greater goal than just earning money.

It would be a damned shame if I ever forget about these things.

Episode 1, Part 1
Episode 1, Part 2
Episode 1, Part 3
Episode 1, Part 4
Episode 1, Part 5
Episode 1, Part 6

Episode 2, Part 1
Episode 2, Part 2
Episode 2, Part 3
Episode 2, Part 4
Episode 2, Part 5
Episode 2, Part 6

Episode 3, Part 1
Episode 3, Part 2
Episode 3, Part 3
Episode 3, Part 4
Episode 3, Part 5
Episode 3, Part 6

Tyan GT20 B5191 1U server

2007-09-06T10:26:00.000+08:00

Bought it yesterday, although the server chassis had to be booked in advance for 2 to 3 weeks.

The server will be co-located in one of HKNet's data centers next week.

Server configuration:
Chassis: Tyan GT20 B5191 barebone
CPU: Intel Core 2 Quad Q6600
RAM: DDR2 1GB
Harddisk: Seagate 7200.10 250GB

Content aware resizing algorithm for images

2007-08-30T11:58:00.000+08:00

This computer graphics research has been greatly hyped lately due to the recent report on Slashdot. It is based upon a simple idea but it achieves amazing effects. Not only can it resize images with the minimum information loss or perceptible distortions, it can also be used to achieve similar effects (but not exactly the same) to image inpainting.

The paper can be read here,
Video demonstration of the algorithm, and... next one is the best
Third party implementation of the algorithm that you can download and hack.

Debugging, profiling JavaScript and detecting memory leaks by source code instrumentation

2007-08-30T11:21:00.000+08:00

I found a paper from Microsoft Research yesterday which describes a reliable way for debugging JavaScript remotely on multiple platforms and... here comes my favorite part... reliably detecting IE6 memory leaks in JavaScript.

Why not use Drip instead? While Drip is useful for catching memory leaks, there are many cases where Drip does not work. For example, Drip 0.5 does not catch memory leaks in FCKeditor.

How about lapsed listeners? The paper has talked about lapsed listeners in section 6.2 but I can't see any solution they mentioned in the paper. Perhaps I've overlooked that.

The paper also talks about remote, multi-client performance profiling and many other nice things that could be of tremendous help in modern web application development. While products like Firebug and Tito Web Studio already provides profiling and basic debugger support, they are browser specific and the performance data applies only to the developer's workstation. It is always possible that a web application runs dramatically differently on other computers that the developer has not expected it to, and that's where the paper's approach can help.

You can read the paper here.

Kernel based virtualization on Fedora Core 7

2007-08-20T01:47:00.000+08:00

Linux kernel running Windows XP (well, the installer, at least) via Intel Vanderpool technology. Hmm... the irony.

The installer runs noticeably slower than VMWare though. Also, there are some show stopper usability bugs with Fedora Core 7's Virtualization Manager which are stopping me from fully installing Windows XP in the VM console... thus forcing me to call the VM engine (QEMU/KVM) from the command line to do the second stage of WinXP installation.

Update: Just found the reason for the slow down. While installing Windows XP on QEMU/KVM, the user must press F5 (instead of F7 as indicated by KVM's FAQ) at the beginning of the installation and choose "Standard PC" as the computer type. The problem is documented here.

Microsoft Seadragon and Photosynth demonstartion

2007-08-10T11:30:00.000+08:00

The first demo (Seadragon) is a ZUI implementation of a photo manager. The second demo (Photosynth) is a 3D photo stitching and modeling software.

Both are amazing stuff.

Moving up in the open source world

2007-06-27T07:39:00.000+08:00

BitAnarch... WT Toolkit... FCKeditor

I've just been hired as a core developer of the FCKeditor project, currently one of the top 10 projects in SourceForge.net. It's a very popular piece of software in the Web 2.0 universe.

Alas, very few people in Hong Kong understand what I'm (and have been) doing. The business culture of Hong Kong simply don't appreciate new, cutting edge technology, seeing it as just "toys".

Amway Pan-PRD project competition - catastrophic failure

2007-06-24T05:07:00.000+08:00

I passed the "self-demo" part but screwed up the final presentation part. The HKUST team that went to the final competition in China is the Fung Shui Design System team. The whole failure was very funny, there was no common reason you can apply to that.

First off, I was not stressed at all - the most common reason. When was the last time you saw me being nervous of anything? It's not that I don't care, but I always have the mindset that it's useless to struggle at the last minutes - so why not relax and see how things turn out? I was very tired listening to people talking all day though. To me, talking and listening is much more tiresome than reading and writing, and even thinking.

Then, the presentation was wrong in the beginning (the judges were not looking for demonstrations in the second part), and there's the difficulty of explaining AJAX, the hundreds of current problems with AJAX app development, the toolkit itself, the theories and algorithms inside, and its business value within 10 minutes to a panel who's not been looking at this specific area recently.

So at the end... the judges looked very puzzled. Which was kind of expected seeing that I used 50 minutes in the FYP presentation at HKUST, and even with 50 minutes I was skipping lots of details. Same with that 115-page FYP report, many details missing. Current problems, others' solutions, algorithms, human factors, businesses, ... there were simply too many things to talk about for 10 minutes.

And to add a further insult to the whole screwy presentation, the judges asked me the practical uses of the project - so the whole time they didn't get anything. I happened to answer by giving them a demo of how to program it, but they were expecting me to answer how businesses could use that. EditGrid, PCMS, FCKeditor, Google Maps, Google Office... it's all in my brain. That TnC company even did the web page for iProA and they are living off AJAX technologies. I had all the answers and more, but sorry, time's up. GG.

Permanent fake job adverts, in America

2007-06-24T04:39:00.000+08:00

Why are there so many job advertisements requiring you to know everything and 20 years of .NET experience? It's not ignorance in the employer's part... don't be so naive.

Now the exact same thing doesn't happen in Hong Kong. But then I can't help but notice the local college graduates seem to have a "single measure to fitness" mindset to job seeking, as if it's HKCEE or HKALE. The real world doesn't work like that - the "fitness" in the employer's mind can often be completely different to what you have in mind. They could be looking for someone stupid, cheap and doesn't complain.

Lunch with Prof. Paul Chu

2007-06-10T17:54:00.000+08:00

I, and other CSE Best FYP Award recipients, attended an award ceremony at the President's Lodge on the 8th of June. I was honored to be able to have a lunch with some of the most famous people in HKUST, such as President Chu and Prof. Roland Chin.

I drank a little bit too much wine in the lunch though. You can see my face was slightly red in the photo, and I was feeling somewhat dizzy.

Things to do...

2007-05-28T21:53:00.000+08:00

29 May 12:30 - ECON115 final exam (Sports Hall)
31 May 09:30 - HUMA099G final exam (LTA)
1 June 12:45 - CPEG appreciation lunch (LG7)
1 June 15:00 - Meeting at TnC Ltd. Office
4 June 17:00 - Deadline for submitting Pan-PRD competition materials
8 June 12:30 - Best FYP Award Presentation
1H June - Submit industrial training logbook (long overdue)

Final examination

2007-05-20T17:16:00.000+08:00

Right... after the FYP presentation comes the final examination of the final semester of my UG life.

WT Toolkit presentation at HKUST

2007-05-17T12:30:00.000+08:00

There will be a presentation of WT Toolkit in Hong Kong University of Science and Technology this Saturday. In addition to demonstrating WT Toolkit, we (i.e. me and Marco) will also discuss the difficulties and pitfalls facing AJAX developers, and we will compare WT Toolkit with other popular AJAX toolkits like Prototype and Dojo.

Date: 19th May, 2007
Time: 17:20 - 18:00 HKT (+0800)
Venue: Hong Kong University of Science and Technology, Room 4480

WT Toolkit broke into SourceForge top 500

2007-05-16T10:39:00.000+08:00

Just in time for me to present my FYP on Saturday.

HKUST research on web technologies

2007-05-15T06:46:00.000+08:00

While I was skimming the proceedings of WWW2007 conference for interesting ideas (e.g. this one, a simple method of adding security to AJAX mashups) today morning, I saw a paper from HKUST:

Exploring in the Weblog Space by Detecting Informative and Affective Articles

The paper describes a method that classifies blogs into various degrees between "informative" and "affective". Informative blogs, like Alex Russell's, dispense useful information that interest the readers. Affective blogs are diaries describing things that mostly interest the author only. High quality blogs (i.e. those that people want to read) are usually informative.

The method shouldn't be treated as an absolute measure of blog quality, however. Lets take a look at a random paragraph from Joel on Software, a popular informative blog:

That's why I'm incredibly honored that they invited me to write a guest editorial about recruiting and internships in this month's issue. Thanks to professional editing, it feels a little bit polished compared to my usual style. I don't think I would write, "Ah, college." I do remember writing, "Get me a frosty cold orange juice, hand-squeezed, and make it snappy!"

Highlighted in red is one of the top feature phrases indicating affective blogs, as described by the paper. Guess what the algorithm would classify the above paragraph, and Joel's blog entries in general? I don't have the software on my hands so I can't test it and get the data, but the above paragraph has a word that is ranked as a top representative feature in the affective category, and none in the informative category. And that's not just an isolated example:

Microsoft finally put Lookout back up for download, but they sure weren't happy about it. ... The story has a happy ending.

A number of years ago a programmer friend of mine worked for a company...

... it wouldn't be such a bad thing to take Air France and change planes at CDG.

Among other things, this week I've been working on the new office design with our architect, Roy Leone [flash site].

Microsoft did the only thing that made sense...

I've been nattering on about this topic for well over 5000 words and I don't really feel like we're getting anywhere.

Thanks to professional editing, it feels a little bit polished compared to my usual style.

I had a chance to visit 7 World Trade Center today...

The "like" example above assumes that there's no word sense disambiguation (or something similar) in their algorithm. Since the "like" in the paper and the "like" in my example has different meanings. But hey, the paper didn't mention WSD at all.

On the other hand, the only informative features I could find from Joel's blog today are "project" and "report". They appear much less frequently than affective features in Joel's blog.

Joel's blog, however, is widely regarded as highly informative by software engineers. It's just that Joel prefers to write his blog entries in an informal and personal style. But anyway, this is still an interesting reading in seeing how computers can attempt to "understand" and filter information these days.

WT Toolkit broke into SourceForge top 1000

2007-05-14T20:46:00.000+08:00

Well, a day of top 1000 isn't too hard to do. BitAnarch broke into top 10 for a few days in 2003. But still, this is good. Considering the 190,000 ranked projects in SourceForge, we're firmly in the top 1% of all projects.

AJAX frameworks are NOT pointless

2007-05-13T14:30:00.000+08:00

This was a response I posted to Slashdot a week ago. Why am I reposting it here? It's because I found my own post back when I was searching on Google today. In particular, I found other bloggers and websites bookmarking or discussing the stuff I wrote. So I guess the stuff I wrote was useful? Then maybe you want to know about it too. So, here it is:

There are many little funny things that just happens when you're coding a web application in JavaScript without a framework/library/toolkit helping you. Unless you're really an AJAX/JavaScript wizard, coding an AJAX-enabled web application on your own and mixing online code receipts is a very dangerous thing to do.

Browser inconsistencies

This is the most obvious one, but only the entry to the rabbit hole. If you are not familiar with the example (maybe not exactly the same, but any AJAX web developer worth his salt should have seen one like that) I give below, then please, PLEASE, do yourself, your fellow developers and your users a favor, resist the urge to hack things together for once, use a mature AJAX framework.

An important part of AJAX is that you need to update what is displayed on the web browser in the client side (by JavaScript), without refreshing the page. This implies that you're very likely to have to create and destroy DOM nodes on the fly. Now, how do you create a radio button in JavaScript?

How about...

var node = document.createElement("input");
node.type = "radio"
node.name = ...
node.value = ...

That's what you would do if you follow the DOM standard. But sorry, this does not work. Try to create a radio button with the above code segment in Internet Explorer 6, you'll get a broken radio button - you can't select it. The correct way to create a radio button by DOM manipulation is described in this MSDN article [microsoft.com]:

newRadioButton = document.createElement("<INPUT TYPE='RADIO' NAME='RADIOTEST' VALUE='Second Choice'>")

Memory leaks

The last one was easy. Do you know you can make a web application that leaks memory like a sieve in Internet Explorer 6 by making a simple circular reference like the following one?

var node = document.createElement("div");
node.someAttr = node;

If you're a good programmer, I might have sounded an alarm in your head right now - any circular references involving DOM nodes in IE6 results in memory leaks that persist after URL changes or page refreshes - unless you use an AJAX toolkit that takes care of the issue for you. Have you assigned a DOM as an attribute value under another DOM node in the past? Yes? Then you'd better check your web application for memory leaks with Drip [outofhanwell.com], now.

What's more, it's not just assigning DOM nodes as attributes that would result in memory leaks, closures in JavaScript can also form circular references and cause memory leaks. What makes closures particularly dangerous is that circular references with closures are not easy to spot. For example, the following code segment leaks:

var node = document.createElement("div");
var clickHandler = function(){};
node.onclick = clickHandler;

Looks innocent enough, but you've already formed a leaky circular reference here. node->clickHandler->node.

For more information about memory leaks under IE6, read these:

Mihai Bazon's blog entry [bazon.net]
MSDN's lengthy and confusing description of the problem [microsoft.com]

The XMLHttpRequest object is not as simple as you think

Much of the magic of AJAX comes from the XMLHttpRequest object (or its ActiveX equivalent, or an iframe, etc.), right? Sure. If you're only doing something simple via AJAX (like, updating the server time), then you can just copy an XMLHttpRequest code snippet from sites like this [apple.com] and hack away, right?

Wrong! Those XMLHttpRequest code snippets are one of the very reasons why people are thinking AJAX as a hack - it sometimes doesn't work! The XMLHttpRequest code snippet given on Apple's site can be broken in commonly encountered situations, and you can simulate that yourself:

Write a simple AJAX web application that retrieves and displays the current server time on a web browser using Apple's code snippet.

Test it yourself under normal conditions. So it works and it's safe to use, right? Let's see...

Change your computer's routing table such that you can have no route to the web server.

Now test your application again in Firefox. Your application should fail. But does it fail gracefully? No. You see an error message in Firefox's error console stating that the XMLHttpRequest object's status attribute cannot be read. If you have coded something to handle AJAX request failures, your handler won't be called.

Why is that happening? It is because, any socket errors happening during an AJAX request will cause the onreadystatechange handler to be called under Firefox, yet the status attribute cannot be read. Reading it causes a JavaScript error and stops JavaScript execution (unless you add a try...catch... block there, but that assumes you already know about the problem so it's moot)! Under Internet Explorer, reading the status attribute in the same situation gives you the socket error code instead. Don't know about these stuff? Please, use a mature AJAX framework.

Performance problems

Coding AJAX applications is just like writing things in C++ or Java - so long as you're using efficient algorithms, your application should run fast, right?

Of course, you are wrong again. Let's say... in some part of your application, you want to concatenate a lot of string fragments together to form a long string in a for loop, how do you do it? How about...

var targetString = "";
for(var i=0;i<someArray.length;i++)
targetString += someArray[i];

That's the way most programmers would think of, intuitively. But the performance of that sucks under Internet Explorer. The correct way to combine strings under JavaScript is to use the Array.join() operation. You can read more about this here [comet.co.il]. The optimization I talked about is also implemented in Dojo Toolkit (kudos to Alex Russell), and I believe any reasonably robust AJAX framework should have it too. Not knowing about such problems, had you hacked together a fairly sophisticated AJAX web application yourself, you would be running into performance hell sooner or later.

Taking 646ms to combine strings still doesn't sound very slow for you, right? There are many more performance traps in JavaScript. Do you know there's a very significant performance difference between the following two code snippets?

First code snippet:

// placing 5000 "Hello World" messages in random positions
for(var i=0;i<5000;i++)
{
var node = document.createElement("div");
node.appendChild(document.createTextNode("Hello World!"));
document.body.appendChild(node);
node.style.position = "absolute";
node.style.left = parseInt(Math.random() * 800) + "px";
node.style.top = parseInt(Math.random() * 800) + "px";
}

Second code snippet:

// placing 5000 "Hello World" messages in random positions
for(var i=0;i<5000;i++)
{
var node = document.createElement("div");
node.appendChild(document.createTextNode("Hello World!"));
node.style.position = "absolute";
node.style.left = parseInt(Math.random() * 800) + "px";
node.style.top = parseInt(Math.random() * 800) + "px";
document.body.appendChild(node);
}

The only difference between the two code snippets is the placement of the document.body.appendChild() line. But if you actually test them out, the second code snippet is much faster, under both IE and Firefox. The performance difference has nothing to do with your algorithms - you just shuffled one line of code around; it has to do with how the browser render the randomly placed DIV nodes. Ever wondered why your hacked together web application is taking half a minute running JavaScript after all the files are loaded?

So, unless you're already a programming god or don't mind spending lots of time solving bugs that you shouldn't have solved; you really, really should use some of these AJAX frameworks if you're making anything fairly sophisticated with AJAX.

Feeling sick today...

2007-05-11T06:26:00.000+08:00

My throat felt a little dry after eating some Bolognese spaghetti (it's just a $25 dish at fast food restaurants, not expensive stuff) for lunch yesterday. I thought that was normal, coz the spaghetti was somewhat spicy, and the soup was spicy too. I went on to attend lessons and meetings as usual. I slept at 6pm that day (yes, you read that correctly, 6pm, I have crazy sleeping times).

I got up by 2am today, and that little dryness I had in my throat turned into pain. Oops? Just what I had done wrong? I certainly ain't overworking myself these days. But I have a presentation to do today, that sucks. :(

hkpcug.homeftp.net goes down for a day

2007-05-10T09:53:00.000+08:00

There's no electricity to my home today due to a routine checkup. Server will get back online tomorrow.

Delayed execution idea scrapped, but the optimizations stayed

2007-05-06T21:15:00.000+08:00

After some more thoughts, delayed execution is found to be stupid. It requires the web developer to change their code to get the benefits, and it sometimes breaks your application.

What's this delayed execution stuff about originally? It's actually a trick to get around browser inefficiencies in rendering DOM nodes with CSS attributes.

Consider the following two code snippets:

// placing 5000 "Hello World" messages in random positions
for(var i=0;i<5000;i++)
{
var node = document.createElement("div");
node.appendChild(document.createTextNode("Hello World!"));
document.body.appendChild(node);
node.style.position = "absolute";
node.style.left = parseInt(Math.random() * 800) + "px";
node.style.top = parseInt(Math.random() * 800) + "px";
}

// placing 5000 "Hello World" messages in random positions
for(var i=0;i<5000;i++)
{
var node = document.createElement("div");
node.appendChild(document.createTextNode("Hello World!"));
node.style.position = "absolute";
node.style.left = parseInt(Math.random() * 800) + "px";
node.style.top = parseInt(Math.random() * 800) + "px";
document.body.appendChild(node);
}

Both code snippets place 5000 randomly positioned "Hello World!" messages in the browser window. The two code snippets differ only in the placement of the document.body.appendChild() line. Running the first code snippet in Firefox can take 1 minute or more, but running the second one takes only a few seconds. The second code snippet provides a more than 10x speedup compared to the first code snippet.

Similar phenomenon can be observed in Internet Explorer also, but only with much more complicated logic, so we'll not go over that. But anyway, the moral of the story is, modifying some CSS attributes (especially positioning attributes) is harmful after the DOM node is already visible.

So what did the scrapped delayed execution idea has to do with these browser weirdnesses? The delayed execution idea was meant to help in batch widget creation and batch CSS style manipulations. e.g. when you're creating 100 widgets in a single pass. It speeds up widget creation or CSS style manipulation by making a common ancestor DOM node of the widgets being manipulated/created invisible before executing the performance sensitive code, and making the ancestor node visible again after execution.

Sounds like a hack - yes it is a hack. It sometimes breaks your application code, it requires you to change your application code to use it. But as shown in the videos, it worked.

Now, the hack is scrapped, before it is even released. And that's because we've got a more consistent way of implementing the same optimization in WT Toolkit, without the need of using hacks.

So what do we have for 0.3.3 now:

Massively increased widget creation performance in Internet Explorer, without needing the developer to change a single line of code.
No performance improvement in Firefox if you don't change your application code... Oops?! But hey, that would be the same if we implemented the delayed execution hack.

But what if you want to make your WT Toolkit application run faster in Firefox? Just pass null as the parentWidget argument to the widget constructor as much as possible, and add the widget to the document tree only after you've done all the CSS manipulations.

Say, if you have

var n = new wtButton(myParent, "Yes!");
n.setAbsolutePosition(x, y);

Then, the optimized version would be

var n = new wtButton(null, "Yes!");
n.setAbsolutePosition(x, y);
myParent.addWidget(n);

Actually, you can perform the manual optimization with WT Toolkit 0.3.2 too.

The art of presentations

2007-05-05T18:23:00.000+08:00

I watched a total of four presentations and did one myself last Friday. Out of the four presentations that I saw, three were group presentations done for course projects, and the other one is a solo presentation done by an engineer in IELM311.

In the group presentations I watched, there was one presenter that was extremely remarkable - remarkably bad and unnatural. Good presentations feel like an old friend talking to you, even though you've never met the presenter beforehand. This guy... he spoke "perfect" English during the whole presentation, more perfect than native speakers - there was not even the slightest pause in his presentation. He just kept talking talking and talking, jumping around mechanically as if those were gestures, with a smile always so wide on his face that he looked schizophrenic.

But aren't these stuff what our English teachers taught? Of course, nobody taught you to deliver your gesture mechanically, yet there's always somebody who goes too far in following those lessons.

The presenter on Friday got me recalling another presenter I saw when I was in a public speaking competition in form 6 - there was another presenter from another top secondary school that acted exactly like him. That presenter also spoke perfect English - with appropriate pauses this time, even. But there was something very unnatural with him - his body was swinging like a pendulum the whole time during his presentation. Looking at him makes you feel like attending a rave party. The judge (who was a foreigner) gave him a very low grade as a result.

What is a good presentation? I've seen good and exciting presentations where the presenter didn't even speak good English (e.g. Tam Wai Ho's presentation in IELM311). The differentiating quality between good presenters and mediocre presenters is their ability to make the audience feel comfortable and keep them thinking instead of falling asleep. When you're seeing a product presentation and you're thinking, "Hey, this product seems amazing, what uses do I have for it? How did they do it? Are there any modifications that I'll need if I were to buy it?", then you're looking at a good presenter. This unique quality cannot be emulated by simply speaking good English (you can even do without that) or having tons of gestures in your presentation, as your English teacher would have taught you. But how did the good presenters do that? I wish I know. But understanding the audience should be the first step, since a good presentation directs the thoughts of the audience.

Where are the good presentations? Apple have them.
http://www.youtube.com/watch?v=HGTX_f3Piko

WT Toolkit FYP Poster on display in HKUST Academic Concourse

2007-05-03T00:50:00.000+08:00

If you come to HKUST, you can find our poster at the "Software Technologies" section of the CS FYP poster displays, in front of Lecture Theatres A and B.

Information wants to be free: 09-f9-11-02-9d-74-e3-5b-d8-41-56-c5-63-56-88-c0

2007-05-02T23:56:00.000+08:00

░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░
░09░░░░░░░░░░░░▒▒▒▒▒▒░░░░░░░░░░▒▒▒▒▒▒░░░░░░░░░░░░
░░░░░░░░░░░░░░░▒▒▒▒▒▒▒▒░░░░░░░░▒▒▒▒▒▒░░░░░░░░░░░░
░░░░░░░F9░░░░░░▒▒▒▒▒▒▒▒░░░11░░░▒▒▒▒▒▒░░░░░02░░░░░
░░░░░░░░░░░░░░░▒▒▒▒▒▒▒▒░░▒▒▒▒▒▒▒▒▒▒▒▒░░░░░░░░░░░░
░░9D░░░░░▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒░74░░░░░░░░░
░░░░░░░░░▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒░░░░░░░░▐▌░░
░░E3░░░░░▒▒▒▒▒▒▒▒▒▒██▒▒▒▒▒▒▒██▒▒▒▒▒▒▒░░░5B░░░▐▌░░
░░░░░░░░░▒▒▒▒▒▒▒▒██▒▒▒▒▒▒▒▒▒▒▒██▒▒▒▒▒░░░░░░░████░
░░░D8░░░░░▒▒▒▒▒██▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒██▒▒▒░░░41░░░██░░
░░░░░░░░░░▒▒▒██▒▒▒████▒▒▒▒▒████▒▒▒██▒░░░░░░░░██░░
░░░56░░░░░▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒░░░░░░░░██░░
░░░░░░░░░░▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒████████░░
░░░░░░██▒▒▒▒▒▒▒▒▒▒▒██████████▒▒▒▒▒▒▒▒▒▒░░░C5░░░░░
░░░░██░░▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒░░░░░░░░░░
░░██░░░░▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒░░░░░░░░░░
░░██░░░░▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒░░░░░░░░░░
░░░░░63░▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒░░░░56░░░░
░░░░░░░░▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒░░░░░░░░░░
░░░░░░░░░░░░░░░░░░░░██░░░░██░░░░░░░░░░░░░░░░░░░░░
░░░░░88░░░░░░░░░██████░░░░██████░░░░░░░░░░C0░░░░░
░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░

So, the DeCSS debacle all over again. Somebody print that on a t-shirt.

http://blog.digg.com/?p=74
http://yro.slashdot.org/yro/07/05/02/0235228.shtml
http://www.chillingeffects.org/anticircumvention/notice.cgi?NoticeID=7180

Why is Google Bot not crawling WT Toolkit website?

2007-04-30T22:31:00.000+08:00

To monitor traffic to WT Toolkit's website, I sneaked in some PHP code that logs down information about incoming visitors from 24^th April - just 6 days ago.

At an early stage of a project, I wasn't too concerned about human visitors (that aren't too many, honestly), I was concerned about the search engine bots. The log file I got indicated that Googlebot would visit my site daily, but it stopped at the main page and did not crawl further. So every day, there's an isolated Googlebot log entry visiting the main page once and didn't do anything else.

Like...

2007-04-25 22:05:58 66.249.66.138 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) /xoops/modules/wtHome/ ref=

That does not make sense, there are plenty of simple links on my front page that any search engine crawler should be able to crawl. But then, these isolated log entries repeated every day, Google just didn't crawl my project website. What's worse, searching for "site:wt-toolkit.sourceforge.net" on Google still gives me the "Generated Javascript Documentation" result, which indicates that Google completely ignored the new project website despite that fact that they have seen the main page a few times already.

While there's some fancy Javascript trickery on my project website (like the project logo), most of the project site is written in traditional PHP/HTML such that search engine crawlers can easily understand it. The project website looks perfectly legible even if you disabled Javascript. What can possibly go wrong here?

I found a tool today, that (claims to be) is able to simulate what Googlebot sees from your website.

"Be The Bot"
http://www.avivadirectory.com/bethebot/#

So I entered "http://wt-toolkit.sourceforge.net" into the tool, and surprise! It says Googlebot sees a completely empty page there.

How could that happen? Immediately I thought of the redirecting index.php I put up in the root directory of WT Toolkit's project website. It only had one line of PHP code (three lines if you count the php opening and closing brackets):

<php?
header("location: xoops/");
?>

I put it there because I installed XOOPS (which is the CMS behind WT Toolkit's project website) under the xoops directory, and not the root directory. I did that for convenience. Going inside "xoops/" would give you yet another redirection, which gets you to the "Home" module's URL "/xoops/module/wtHome/".

Was Googlebot not able to process the redirection? It seems to be able to follow the redirections, otherwise it wouldn't be visiting "/xoops/modules/wtHome/" in the log file. Be The Bot's simulation also left the same log entry in my site log file, however.

So I entered the URL without redirections to Be The Bot: http://wt-toolkit.sourceforge.net/xoops/modules/wtHome/

This time, it displayed the project website correctly, albeit without the images.

Something was definitely wrong there. The log file indicates that Be The Bot was redirected to "/xoops/modules/wtHome" successfully, yet it couldn't retrieve the HTML correctly. Without redirection, the correct HTML content was retrieved. XOOPS might be part of the problem here, but I'm not sure.

Anyway, this means I have to restructure the project web site a bit so that the main page can be retrieved without redirection. This is not difficult... Done. No redirections for the main page now.

Let's see if Google could crawl it correctly tomorrow or a few days later.

WT Toolkit 0.3.3 Performance Optimizations

2007-04-30T05:40:00.000+08:00

A problem plaguing WT Toolkit ever since its birth is performance. WT Toolkit 0.1.x and 0.2.x felt slow all the time because of the garbage collector running in the background. 0.3.0 eliminated the background garbage collector and yet we kept it automatic so the programmer doesn't have to care about lapsed listeners (well, most of the time).

But at 0.3.2, our performance is still bad compared to other popular toolkits like Dojo Toolkit and Qooxdoo. Widget creation latencies increase linearly in a very quick manner as the number of on-screen widgets increases. The effect isn't very noticeable under Firefox, but WT Toolkit 0.3.2 definitely felt slow under Internet Explorer 6 or 7.

Well... not anymore for the upcoming WT Toolkit 0.3.3! Even though I've already submitted my FYP final report, new work has begun on performance optimizations! Yeah, baby!

How much have we optimized? Let's see what a little trick called "delayed execution" (available in WT Toolkit 0.3.3) can do...

Before optimizations:

After optimizations:

As a result of the work on performance optimizations, I left the work on WT Toolkit website to Marco. He couldn't complete it on 29th April because he had other academic work to do at that time. But anyway, we're having steady progress on WT Toolkit's website, we'll be seeing more and more amazing things as time goes on. :-)

Deleted

2007-04-29T03:40:00.000+08:00

Group norm 的確係不好挑戰的

算, 我係 hea 的. 我無野好講.

FYP Poster for WT Toolkit

2007-04-27T23:50:00.000+08:00

Drawn and color-printed out last night with GIMP and Inkscape. Marco pasted the individual A4 sheets on the poster board and handed it in to CSE department.

Plans for WT Toolkit

2007-04-25T09:13:00.000+08:00

Now that we are fairly feature complete, time for some publicity.

27/4/2007 - FYP Poster
29/4/2007 - Completed WT Toolkit Website
1/5/2007 - Submit WT Toolkit to Ajaxian
1/5/2007 - Submit WT Toolkit to freshmeat
1/5/2007 - Submit WT Toolkit to Open Directory Project
13/5/2007 - Visual programming demo for WT Toolkit
19/5/2007 - FYP Code CD (Not sure what contents are needed, probably a Linux LiveCD)
21/5/2007 - FYP Presentation (schedule - we are group DE3)

Watched Michael and John's FYT pre-presentation today

2007-04-23T19:10:00.000+08:00

Why did I go to the presentation? That's because I talked to Michael about his research today morning and I found it interesting.

The topic was implementing an efficient DHT on an ad-hoc mobile network. Efficient DHTs for fixed-line, broadband Internet are already there, like Chord and Pastry, and everybody is using those knowingly or unknowingly. Michael's research is about how to make DHTs efficient on ad-hoc mobile networks, which is much harder than implementing DHTs on top of our everyday IP network. Some difficulties include:

1. Message routing. Mobile nodes do not and should not have fixed routes like our desktop computers. Although routing on the physical network can be partially solved by things like AODV, you still have to make sure the hops on the DHT's overlay network are efficient. e.g. assuming you've got perfect data routing in the physical network, it's still useless if one of the DHT hops goes to another country with a 12-hour timezone difference - your message will be hopping across many many many nodes in the physical network for just one DHT hop.

2. Bandwidth overhead. (??) I don't know how bad the problem is since I haven't seen the simulations myself. Probable causes I've heard are AODV-style flooding and Bloom filter inefficiencies. Gnutella-style implementations were mentioned for the audience to point and laugh at, I guess.

One of the related papers to M and J's work:
http://www.cs.ucsb.edu/~ravenben/publications/pdf/idlp-comsware07.pdf

Now what's Michael and John's proposed solution... They proposed a DHT that's organized in a tree-like fashion, instead of the ring/skip list type seen in Chord or Pastry. The root node in the tree is called a "landmark", which should have a fixed location and has no extra hardware resource requirements when compared to other nodes. Their algorithm takes care of the physical routing as well so there's no need for AODV flooding or playing with Dijkstra's algorithm as in SrcRR. No AODV, no route request flooding, less bandwidth overhead. Bloom filters are used in narrowing/selecting paths in the tree, which is very intuitive and easy to understand (just a simple trick with bits, with the hard probability maths done for you 30 years ago), despite the seemingly cryptic name.

Prof. Gary Chan asked lots of questions during the presentation, he had a very sharp sense for things that seemed to be "strange" or inefficient. The object duplication algorithm (put in there to make p2p swarming possible) in John's presentation was one of the quirks Gary spotted, the algorithm seemed like a placeholder, I guessed it shouldn't be too hard to correct though.

So what I've got from the seminar... let's see
1. Revision of some old algorithms (Bloom filters... I almost forgot them completely, never used them once in the past few years), learned some new ones, and some new problems.
2. The 40 minutes presentation time I've got for my FYP is preciously short. Michael and John's presentation went for like 1.5 hours, and they were still missing on some details.
3. I need to keep my audience interested by doing demonstrations, with both WT Toolkit and WT Toolkit's competitors.

Regular Expressions - how good theory is ignored in popular software

2007-04-23T09:14:00.000+08:00

You think the regular expression implementation in Java, Perl, Python, PHP, Ruby and PCRE (which is a C library) should have been refined many many times and thus highly optimized? Think again.

http://swtch.com/~rsc/regexp/regexp1.html

The title of the article is "Regular Expression Matching Can Be Simple And Fast", but what's more interesting is the subtitle - "(but is slow in Java, Perl, PHP, Python, Ruby, ...)". Slow, how slow? Look at the first graph of the article, for some pattern matching inputs, Perl 5.8.7's built-in regular expression matching is millions times slower than a 40-year-old algorithm.

How can that happen? Well... it could be argued that the expression used in the example is a pathological case. But is it a pathological problem in theory? i.e. not belonging to P, or belonging to P with a very large exponent? Well, obviously not. Otherwise, the 40-year-old algorithm wouldn't be able to perform the matching quickly as well.

What actually happened here was this... all the popular programming language developers (Java, Python, Perl, PHP, etc.) copied/borrowed their implementation from a popular extended regular expression matching algorithm that was known to be "fast enough", but not known to be provably fast. 40+ years of theories of finite automata went into the trash bin when programmers (including the guy who invented the correct algorithm 40 years ago!) needed to release softwares fast and neglected to spend time to think about the mathematics behind.

The regular expression engine that the article's author described was only a very simple one, however. Can it be expanded to processing modern extended regular expressions without going into the same performance hell of Perl, PCRE, Python, etc.? The author gave some justifications that it could, but he was very light on the details. Even if he has missed out some details that makes his proposal infeasible, however, it still stands that the regex engines we're using every day are far from optimal.

Biometrics a fad?

2007-04-23T07:35:00.000+08:00

How secure is it to use your fingerprint as an authentication token? Much research has been done to that, so it must be secure, right?

But wait a moment... you leave your fingerprints everywhere, every day. It's pretty much public information. And using public information as a secret key sounds like a dumb idea, doesn't it?

Yup... it's dumb. Everybody can crack a fingerprint scanner with a printer, transparency slides, PCB etching tools, and any moldable plastics. It's at its heart security by obscurity. And it's remarkable how much bullshit went into that "unbreakable door lock" in the video. Using moisture as an authentication condition?! On come on, is moisture really so scarce or secret on Earth? Now what's next? Iris scanners? Your iris pattern can be captured everywhere, in 3D, even... it might be a little bit more difficult to capture and reproduce, but it's public information, nonetheless. If what they are betting on is the resolution of cameras (which can definitely be improved as time goes on), then they're relying on security by obscurity.

It's remarkable how far snake oil technologies can make into the market, government institutions, and even academia.

By the way, the video rocks! It feels like reading an early issue of the Phrack magazine (much of the hacks don't work anymore, of course. But wait... the fork bomb still works ) or some of the classical papers/theses (like, Chord). Easy to read, concrete procedures, concrete results, and profound implications.

WT Toolkit listed under opensource.hk

2007-04-23T06:13:00.000+08:00

Just saw this when I was searching in Google. Good to know there are people who know my project exists, and there are other people doing the same thing as me.

URL: http://opensource.hk/opensrcproj

Among the projects, the only other ones I can recognize are CK-ERP and RMSS. CK-ERP's author, C.K. Wu, has worked on the project for many years. He posted many advertisements in local newsgroups. Sadly, there's rarely any public replies to him. There should have been quite a number of people talking to him privately though, as ERP systems are generally very expensive and have major impact on business.

Martin Kou's Blog

Web Services in Python (Part 2) - Serving Static Files

2011 in a one word for me...

Web Services in Python (Part 1) - You Only Need 3 Lines Of Python Code

I bought a domain at Namecheap today, because GoDaddy supported SOPA

The Easy Way Out

The Easy Way Out

Salesmen and Bean Counters

Ethics

Present State of HTML5 in Mobile App Development

Two Divergent Roads

Node.js for Client Side

Back to the Browser

Two UI Libraries

Debugging and Making Changes

The Present State

How to make the Push Notification prompt reappear during iOS app development

Now for something completely different...

Some predictions and wishlist for the coming year

An alternative workaround for Mobile Safari's innerHTML problem

A Probable General Solution

AFeature - Interactive JavaScript console for mobile HTML5 applications

How to solve "assembler for architecture ppc not installed" errors after installing XCode4

矽巷啁啾：朱克伯格以外的事（三）

矽巷啁啾：朱克伯格以外的事（二）

吃自己狗糧 了解客戶感受

Some performance tests on gevent and libevent latencies

矽巷啁啾：朱克伯格以外的事

創業者常有荒誕思維

AirBnB碌爆卡才成功

A Review of 2010

矽巷啁啾：矽谷的融資環境（二）

種子資金提供生存時間

藉收購、擴市場網羅人才

矽巷啁啾：矽谷的融資環境

投資者分三類

How to get 64-bit Firefox 4b7 for Mac today

How much faster is your iPhone's CPU compared with your computer?

Really short way to generate a secure password in Python

Foursquare considered harmful - DON'T USE [Updated w/ 4sq's reply]

List of hidden commands in UNIXKCD

Me on the TV

The really interesting part of Google Wave - Operational Transform

CKEditor 3.0 released!

Me on the newspaper

FCKeditor is looking for 2 experienced JavaScript developers in Hong Kong and China

The irony of it all

Yet another boost to JavaScript execution performance - Squirrelfish Extreme

BarCampHK 2008

My opinion piece on web browsers in PC Market HK

Real time image editing in JavaScript, with Firefox 3.1

Watch out for when your iPhone 3G refuses to receive calls

A taste of what's coming in CKEditor 3.0

Things the iPhone 3G technical specs don't tell you...

How to properly fix the Mac OS X ARDagent security hole

More devils in the details

The new flat itself, and the little details...

The second to fourth days of moving...

Things to write about in the coming days.

First day in the new flat - continued

The first day in the new flat

Long time no blog - moving home

New year presents from Opera

GWT's solution to the JavaScript dynamic memory leak problem.

Got Mac OS X Leopard today.

WriteArea: Use FCKeditor everywhere you want!

Introducing WriteArea

Using WriteArea

Step 1: Switch to "Edit HTML" in Blogger, and activate the plugin by right clicking on the text area.

Step 2: Write your message in the WriteArea dialog that pops up.

Step 3: Click "Save" in the WriteArea dialog to save the HTML code into the blog site's text area. If you want to edit your post again just right click on the text area and activate WriteArea again.

Step 4: Publish your blog post when you've done editing it.

Getting WriteArea

Early history of the infotech industry - Triumph of the Nerds

Tyan GT20 B5191 1U server

Content aware resizing algorithm for images

Debugging, profiling JavaScript and detecting memory leaks by source code instrumentation

Kernel based virtualization on Fedora Core 7

Microsoft Seadragon and Photosynth demonstartion

Moving up in the open source world

吃自己狗糧　了解客戶感受