Welcome to the blog of Marcel Oelke from Frankfurt, Germany.

Me on Soundcloud

Soundcloud.com is a really great site for producers and music lovers. You can listen to some of my tunes using their player here:

Updated: RR

MoboKoo

Fameball

Comments are always welcome, here or on soundcloud.
More of my productions here: http://soundcloud.com/marceloelke

Plusser.de ist online

Seit kurzer Zeit ist das Projekt (Plusser), an dem ich arbeite, mit dem ersten Release online. Plusser.de ist ein Portal, das über den schlichten Community-Gedanken hinaus geht. Plusser.de soll die Startseite ins Internet für alle ab ca. 40 Jahren werden, da Plusser.de Spaß, Kommunikation und Lernmöglichkeiten miteinander verbindet. Obwohl im Zentrum des Portals der Erfahrungsaustausch steht, bietet das Portal u.a. zusätzlich aktuelle Neuigkeiten, einen E-Mail-Dienst und die Möglichkeit alte Freunde wiederzufinden oder neue, interessante Menschen kennenzulernen. Derzeit tummeln sich noch ein paar kleinere Bugs auf der Seite, aber ich denke diese werden wir bald in den Griff bekommen. Wer Lust hat sich mal anzuschauen was ich im “echten Leben” so mache sollte einen Blick auf www.plusser.de werfen ;).

Besonders interessant war für mich die Entwicklung des E-Mail-Clients und der Einsatz des Zend Frameworks, auf dem einigeTeile der Website basieren. Mittlerweile bin ich ein Großer Befürworter des ZF. Wir haben sehr viel Zeit durch den Einsatz des Frameworks gespart.

Jeder Benutzer kann sich übrigens bei der Registrierung entscheiden ob er sich mit seiner eigenen E-Mail Adresse anmelden möchte, oder ob er eine kostenlose Adresse von Plusser.de haben möchte. Momentan sind mehrere Web und E-Mail Server für Plusser.de im Einsatz, wobei ich denke dass wir schon bald aufstocken werden müssen. Wir verwenden übrigens mehrere MySQL 5 Server mit Replikation sowie PHP5. Für das Load-Balancing sind zwei BIG-IP’s von F5 zuständig.

So, nun aber back 2 work …

Who is the Subversion King in your Company?

Have you ever wanted to know who’s the top committer in your company?
In my previous company we etablished the term “CVS King”, a title comparable to “Employee of the month”. The developer with the most cvs commits was the “CVS King of the month”. We determined who was the “CSV King” using commit emails that were sent to all developers on each cvs commit.
Two years ago we switched to Subversion, so now we’re talking about the “Subversion King”. Naturally all this is anything but serious ;)

Anyway, today i programmed a little php script that uses a different approach to determine who is the “Subversion King of the Month”. It’s counting the line delta directly from the svn repository using svnlook. So the developer with the most lines added to the repository (not the most commits) is the number one.
Here’s some example output (names are anonymized):

--- Most productive users for 11/2007 ---
1   usera                6319      42.47%
2   userb                5797      38.96%
3   userc                1990      13.37%
4   userd                 773      05.20%
--- Most active commiters for 11/2007 ---
1   usera                  47         47%
2   userb                  34         34%
3   userc                   4          4%
4   userd                   2          2%
-----------------------------------------

The script reads all commits (revisions) for the current month, counts the line delta (how many lines have been added/removed) and the amount of commits.
You can view the highlighted source here or download the script here. Use it on the command line like that:
php -f /path/to/svn/repository

I know that the number of lines comitted may not be the one and only criteria to measure the productivity of a developer. But its an indicator to start with.
So the question is: Are you the “Subversion King” in your company/project?

P.S.
Again, please, dont take this serious ;)

Record mouse movement using Javascript and AJAX

I just found some piece of code i created one or two years ago. The “movelogger” records the mouse movement a users does on a web site. Just before the user leaves the current page, the recorded data get posted back to the server using AJAX.
The cool thing is that you can “replay” these movements afterwards. The movelogger records clicks on links and other elements. In replay mode this events are fired in the exact same order as they have been recorded.
That way it would be possible to record a websesion (the click-flow) in a heavy AJAX based application. It would even be possible to record keyboard strokes and other type of events.

Check out the little demo of the movelogger here.

This technique may be usefull or not. But some use cases could be:

  • Instead of eye tracking, use mouse tracking.
  • Analyze the usage of ajax enabled websites.
  • Spionage and other Bad Things™ (not recommended).

The whole thing is coded in javascript using Prototype and script.aculo.us with some php code on the server side.

The data recorded on the demo website is stored only in the php session on the server and gets deleted automatically, soon after you close your browser. But theoreticaly it would be possible to store that data in some kind of database for further analysis.

Please let me know what you think about this. Do you have any ideas for other uses of this technique ?

Random Timestamps in MySQL

Have you ever needed a random timestamp in MySQL? For example to create demo data programmatically? Here’s my solution:

SELECT FROM_UNIXTIME(
  FLOOR(
    UNIX_TIMESTAMP('2007-01-01') +
        RAND() *
	(UNIX_TIMESTAMP('2007-01-03')-UNIX_TIMESTAMP('2007-01-01'))
    )
) as random_timestamp;

Or if you prefer a function:

CREATE FUNCTION random_timestamp (start TIMESTAMP, end TIMESTAMP)
RETURNS TIMESTAMP NOT DETERMINISTIC
RETURN FROM_UNIXTIME(
  FLOOR(
    UNIX_TIMESTAMP(start) +
    RAND() *
    (UNIX_TIMESTAMP(end)-UNIX_TIMESTAMP(start))
  )
);

mysql> select random_timestamp('2007-10-01', NOW());
+---------------------------------------+
| random_timestamp('2007-10-01', NOW()) |
+---------------------------------------+
| 2007-10-05 23:07:11                   |
+---------------------------------------+
1 row in set (0.00 sec)

How to name something that imports and exports ?

Simple Question:
When a script/function/class exports something, I name it “exporter”.
When it imports something, I name it “importer”.
How do I name it when it does both?

What do you think? Write your thoughts into the comments please. Thanks!

New minimal music

Hey guys

Here are some of my latest tunes. I guess its something between minimal, electro and techno. None of these is finished 100% yet, but you’ll get the idea.

http://pure.rednoize.com/tunes/

By the way, the picture in the background is taken in my favorite club, the water gate in Berlin.
I appreciate any comments !

Thanks
puRe

MySQL Performance: Use counter tables

I guess many of you know, that using SELECT count(*) FROM table is problematic and slow when using Innodb tables.
This actually only applies to COUNT(*) queries without WHERE a clause as mentioned in the MySQL Performance Blog.

But if you got some slow count query in your application the best way to increase its performance is to replace / remove it.

So if you are going do to "SELECT count(*) FROM products" the best way, is to have a separated table
that stores the number of products. If you're inserting a row increment the counter, if you're deleting a row, decrement it.

Here is some example:
CREATE TABLE counter( number_of_products int(10) DEFAULT '0' NOT NULL);

Increment when you're adding a new product to the products table:

SQL:
  1. UPDATE counter SET number_of_products = number_of_products +1;

Decrement when you're removing a product:

SQL:
  1. UPDATE counter SET number_of_products = number_of_products -1;

In one of my applications i have the case that i have many reads on a table, but just a few updates / writes to the table. The count was in my case not related to a table, but specific to a row of it.

Example:

SQL:
  1. CREATE TABLE user (
  2. user_id int(10) DEFAULT '0' NOT NULL,
  3. username int(10) DEFAULT '0' NOT NULL,
  4. ...
  5. number_of_purchases int(10) DEFAULT '0' NOT NULL,
  6. );

Scenario:
Just imagine that the number of purchases a user has made is somehow complicated to calculate and query. That's why we store the number directly in the user table.
We also have a website that produces a lot of reads to this table column, but (unfortunately) its quite seldom that a user purchases something. So using this complicated count query in each website impression would be slow.

What comes handy here is using MySQL 5 triggers.
With triggers we can calculate the number_of_purchases each time a purchase is made by a user and update the users number_of_purchases column.

Example:

SQL:
  1.  
  2. DELIMITER //
  3. CREATE TRIGGER trg_update_user_purchases after UPDATE ON purchases
  4.  FOR each row BEGIN
  5. call count_and_update_user_purchases(new.user_id);
  6.  END;
  7. //
  8. DELIMITER ;
  9.  
  10. DELIMITER //
  11. CREATE TRIGGER trg_insert_user_purchases after INSERT ON purchases
  12.  FOR each row BEGIN
  13. call count_and_update_user_purchases(new.user_id);
  14.  END;
  15. //
  16. DELIMITER ;
  17.  
  18. DELIMITER //
  19. CREATE TRIGGER trg_delete_user_purchases after DELETE ON purchases
  20.  FOR each row BEGIN
  21. call count_and_update_user_purchases(new.user_id);
  22.  END;
  23. //
  24. DELIMITER ;
  25.  

So each time a records in the purchase table is inserted, updated or deleted the user table gets automaticly updated.

Your application gains the performance from selecting a static int field instead of executing complex count queries. Additionally your application is not bloated with unnecessary code to increment and decrement the count column, because the trigger does this automatically for you.

MD5 Hash Database Update - Supports SHA1 Now

md5.rednoize.com - md5/sha1 hash databaseMd5.rednoize.com has been updated yesterday. The web site features a new design and has gained support for sha1 hashes. The interface has been ajaxified and the amount of hashes stored in the database has been increased again. The database stores now about 35,000,000 md5/sha1 hashes with their plain text counterpart. And md5.rednoize.com got a new logo as well ;)

The "URL API" has not been changed. You still may use the parameter "p" for plain text and "xml" for xml output. The new optional url parameter "s" defines the search mode (currently supported are md5 and sha1).

I would greatly appreciate your feedback ;)

MySQL: Collation matters when using unique indexes

When using a uniqie index on a text field in mysql, the column collation setting is very important. The collation settings of a column does not only affect sorting and comparsion, but also unique indexes. So you can not insert "a" and "A" into a table that has a unique index on a column that has a case-insensitive collation. The mysql manual about collations: "A character set is a set of symbols and encodings. A collation is a set of rules for comparing characters in a character set."

Here is an example:
The column text in table text1 has a case-sensitive collation (_cs suffix), the column in text2 has a case-insensitive collation (_ci suffix).

CODE:
  1.  
  2. CREATE TABLE text1 (
  3.   `text` varchar(50) character set latin1 collate latin1_general_cs NOT NULL default '',
  4.   UNIQUE KEY uniq_text (text)
  5. ) ENGINE=MyISAM
  6. ;
  7. INSERT INTO text1 (text) VALUES ('a');
  8. Query OK, 1 row affected (0.00 sec)
  9.  
  10. INSERT INTO text1 (text) VALUES ('A');
  11. Query OK, 1 row affected (0.00 sec)
  12.  
  13.  
  14. CREATE TABLE text2 (
  15.   `text` varchar(50) character set latin1 collate latin1_general_ci NOT NULL default '',
  16.   UNIQUE KEY uniq_text (text)
  17. ) ENGINE=MyISAM
  18. ;
  19. INSERT INTO text2 (text) VALUES ('a');
  20. Query OK, 1 row affected (0.00 sec)
  21.  
  22. INSERT INTO text2 (text) VALUES ('A');
  23. ERROR 1062 (23000): Duplicate entry 'A' for key 1
  24.  

Constraints are also affected by collation:
The queries on table text1 give different results (a and A), the two queries on table text2 result in the same row twice (a).

CODE:
  1.  
  2. SELECT * FROM text1 WHERE text = 'a';
  3. +------+
  4. | text |
  5. +------+
  6. | a    |
  7. +------+
  8. 1 row in set (0.01 sec)
  9.  
  10. SELECT * FROM text1 WHERE text = 'A';
  11. +------+
  12. | text |
  13. +------+
  14. | A    |
  15. +------+
  16. 1 row in set (0.00 sec)
  17.  
  18. SELECT * FROM text2 WHERE text = 'a';
  19. +------+
  20. | text |
  21. +------+
  22. | a    |
  23. +------+
  24. 1 row in set (0.00 sec)
  25.  
  26. SELECT * FROM text2 WHERE text = 'A';
  27. +------+
  28. | text |
  29. +------+
  30. | a    |
  31. +------+
  32. 1 row in set (0.00 sec)
  33.  

You can display all available collations using

SQL:
  1. SHOW COLLATION;

or

SQL:
  1. SHOW COLLATION LIKE 'latin%';