Auto checking mysql tables and fixing them

mysqlcheck -u root -p --auto-repair --check --optimize --all-databases

Stripes in Japanese UTF-8

I have finally got working with stripes and I must say I love it (So Far) I know not much beats the beauty of ROR but having come from a PHP background I must say stripes seems to make much more sense (to me anyway)

As always when I start a new project in something new I am plagued by language issues as I work mostly in Japanese. But these issues have been smooth and easy to fix.

First Tomcat - I needed to add all of the language filtering and settings that tomcat suggests.

I found details of this on this tomcat wiki

On my DB connection string for JDBC in mysql I needed to use the following jdbc:mysql://127.0.0.1:3306/dvdstarbox?characterEncoding=UTF-8

then finally I needed to convert my UTF-8 based properties in StripesResources.properties.org to ASCII using the following command in my terminal.

native2ascii -encoding UTF-8 StripesResources.properties.org StripesResources.properties

which I found in this helpful post by Cagan

Hopefully in the next couple of weeks I will have my first App done in stripes finished and am already planning the next one.

Sphinx Search Results

After spending over a week on getting sphinx working nicely with our set up, I managed to reduce our query code from over 300 lines of php code using complex queries with multiple joins etc to using less than 100 lines of streamline code. For maintainability this is infinitely better so a thumbs up there.

In the search I am grabbing multiple categories for products, adding parameters for pricing and paging.

For speed on the queries which were taking time - we were searching 100,000 products in multiple fields, here are some of the speed results I noticed. More specific results to our categories we noticed the old mysql was faster as mysql indexing and custom queries were set up this way.

Search - Old time in seconds - sphinx time in seconds
IH - 7.0302891731262 - 0.11172699928284
電線 - 0.017904043197632 - 0.16618394851685
ホワイト - 2.6403949260712 - 0.33370304107666
LED内臓 - 2.8766248226166 - 0.1538360118866
AS-E22T - 0.2899730205535 - 0.096461057662964
エアコン - 7.1297919750214 - 0.14638090133667
照明 - 8.5626368522644 - 0.54876399040222

On the products where searches lagged the difference was huge up to 70 times faster, on those where searches were already reasonable the difference was either un-noticed or even slower with Sphinx. Mysql was still needed to get the data for each of the products and time to access the Sphinx search daemon will always have some overhead.

But overall this is an amazing tool that has made the search process far more elegant and blazingly fast on some of our previously under performing search criteria. I look forward to seeing how it performs when our db gets up to 200,000 products.

Bind settings zone example

Oh how I hate setting up my bind server, but maybe that is because I make it up as I go along, well this time it went a bit smoother, here is my zone file

$ttl 38400
mackstar.net. IN SOA ns1.mackstar.net. richard.mackstar.com. (
1242737840
10800
3600
604800
38400 )
mackstar.net. IN A 203.189.43.247
www.mackstar.net. IN CNAME mackstar.net.
mackstar.net. IN A 203.189.43.247
mackstar.net. IN A 203.189.43.247
mackstar.net. IN NS ns1.mackstar.net.
mackstar.net. IN NS ns1.mackstar.net.
mail.mackstar.net. IN MX 1 mackstar.net.

This works like a charm

Cucumber - yes this could be an amazing tool, how could this integrate to other language like PHP?

I have just started to play with cucumber and I must say I am extremely impressed. Finally a strong bridge between a clients wishlist and test driven development (behaviour driven developement in this case).

Here is an example feature scenario

Feature: Shopping cart
To be able to shop on the internet
As a customer
I want to be able to see the sum of products I wish to buy

Scenario Outline: Add item to cart
Given I have added item1 into the cart
And I have added item2 into the cart
When I press "View Cart" button
Then the sum should be output on the screen

Examples:
| item1 | item1_price | item2 | item2_price | total |
| Hair Spray | 30 | Dreadlock wax | 50 | 80 |
| Cucumber | 100 | Cabbage | 30 | 130 |

This then can be turned into testable code using webrat, rspec and selenium etc, then the development can begin with cucumber telling you line by line what you need to do next to make your scenario's and features complete. This is pretty amazing and may just help me create a solid workflow. But interested how it would work with PHP and Java, hmmm...??

MySql error using JDBC

Value '0000-00-00' can not be represented as java.sql.Date

This problem was easily fixed by adding

?zeroDateTimeBehavior=convertToNull

To the connection url (Bloody Java! -although I confess I kind of like it...)

Converting half width katakana to full width katakana in ruby

NKF.nkf("-wX", "ｶﾀｶﾅ")