While I thought moving the data folder to another new location is quite simple, but then when restart the mysql I keep getting unusual error. Then I realized on Ubuntu, there is one service called AppArmor that will restrict the access of other service.
Steps for moving the folder.
$sudo /etc/init.d/mysql stop $sudo su $mv /var/lib/mysql /newlocation/mysql $vim /etc/mysql/my.cnf
Change the datadir to the new location, default one is datadir=/var/lib/mysql.
Now modify the AppArmor, this is the part which I totally missed it!
Add the new location into for restriction clearance
/newlocation/mysql/ r, /newlocation/mysql/** rwk,
Restart the services that involved, apparmor and mysql. Done
/$/etc/init.d/apparmor restart $/etc/init.d/mysql restart