SQLite and MongoDB had a baby

And I call it.... MongoLiteDB

I'm probably infringing on some trademarks and all sorts of proprietary things, but that's the name I gave it over at GitHub. If that's a problem, then I expect the affected parties will contact me and I will be accommodating.

What is it? If the name isn't clear, it combines some of the strengths of the two solutions while conveniently ignoring requirements I don't care about, namely performance. It provides a very simple to set up, simple to use database.

SQLite is a great lightweight database solution. It uses a single file to store and retrieve all entries and therefore does not require an external service. It does however require a schema, which can be tedious to maintain and update in a rapidly changing project. MongoDB is a great schema free database. Without having to manage a schema, I can hack away to my heart's content. It does however require an external service. It isn't what I would call a lightweight solution.

In SQLite, a great deal of performance gains result from the structured nature of the data. In MongoDB, the external service works all kinds of magic in order to provide a performant schema free solution. Without either of these, performance probably sucks. (I haven't bothered to benchmark it.)

I decided I sometimes just didn't care about performance. In my simple applications, I don't expect to have significant traffic. I don't even think anyone but me will use the service. I want a solution that is as frictionless as possible during development and prototyping. Performance is my last concern. And so MongoLiteDB was born.

I probably spent more time building MongoLiteDB than it would have taken me to maintain my SQLite schema. I mostly built MongoLiteDB because it was something new and exciting. Hopefully myself and other can get some use out of it.

Anyway, read more about it over on GitHub. It's pretty raw still. The code needs comments. The documentation is nonexistent. It's not even packaged as a Gem yet. If it's interesting to you, please let me know. Leave comments on this blog. Fork the project. File Issues. I think this would be a fun project to expand upon and interest from the community will be a great motivator to do so.

Macro Tubes, Canon T3i and Focus Stacking

So I got a new toy this week - Macro Extension Tube Kit Compatible with Canon Cameras.  Essentially it's a set of spacers that tweak the focus range of standard lenses to allow them to shoot macro more effectively.  They can be a little clunky to use, as they are literally just a spacer between the lens and the camera.  This means you lose any connectivity between camera and lens - aperture control and autofocus are lost.  But they cost less than $10 shipped, so I figured they would be fun to play around with.  The alternative was to invest in a macro lens, which I can't justify.

Anyway, they arrived on a particularly boring day.  Construction in our neighborhood meant a planned power outage for half the day.  Seeing as I am studying Electrical and Computer Engineering, I didn't have much to do without Electricity.  So I got to play with them for a few hours, figuring out how to do everything a little more manually than usual.

So I took a few random shots:

Close up of my computer monitor.  Each pixel is composed of 3 little bars - red, green, and blue - which can be see quite clearly.  This was zoomed in a bit from the original image and cropped.

A close up of my tool case. 

The tag on my son's stuffed animal.

They were some pretty random images, mostly the result of thinking "What could I take a close up picture of next?",  but I was pleased with the clarity of using the macro tubes.  They do create a pretty narrow depth of field, so manual focus is tricky, but it's fun to play around with.

One interesting subject I have been looking at is focus stacking.  When you have a narrow depth of field, you can take multiple pictures of a subject, at different focuses, and later combine them into a complete image, with a greater depth of field than any of the originals.  So I gave it a shot by taking a picture of my wedding ring sitting on top of a white piece of paper.  The software I used was Enfuse, which is a little tricky to use, but once setup it does a great job of pulling the whole image together.

The middle of the paper comes into focus, as well as the bottom edge of the ring.

The paper is in focus in the top right of the image.  The bottom left is out of focus.  The ring is completely out of focus.

The bottom left of the paper is in sharp focus, and a good part of the far side of the ring is in focus.

The paper is starting to move out of focus, as is the far side of the ring.

The paper is out of focus, and the near side of the ring is coming into focus.

The near side of the ring is in sharp focus.

And the final result:

Notice how the paper, the far side of the ring, and the near side of the ring are all in focus.

I used pretty conservative and standard setting with Enfuse.  Some things are more in focus than others, which I could probably fix by tuning Enfuse a little more.  I was very impressed by how well Enfuse combined the images on it's own.

Overall, the macro tubes are fun to play around with.  Probably worth the $10 to have on hand in case you ever feel an artistic need to take macro shots.  Enfuse and focus stacking are definitely something I'm going to have to play around with more in the future.

Using Raphael.js and Backbone.js together

I'm a big fan of Backbone.js. It's lightweight enough to throw into any webapp I'm building, large or small. At the very least it gives me some guidelines on how to organize my code. In larger apps the synchronization with the backend makes my life a lot easier.

An app I'm currently working on makes heavy use of Raphael.js, a great SVG library. In order to unify my handling of user events, I extended the Backbone.js library a little. It's a simple change, but I really like what it did to the organization of code.



As you can see, my goal was to be able to address user interaction with Raphael object in the way I can address interaction with the DOM. DOM events can be specified under events via a "<selector> <event>" syntax, so one can do ".button click" or "#myElement customEvent". Since Raphael abstracts out the DOM from the SVG objects, I wanted to preserve the Raphael events but be able to throw them in with DOM events. So the delegateRaphaelEvents method converts Raphael events to DOM events in the form "raphael:<object name>:<event>".  The assigned callback function gets passed two objects, a jquery event object and a raphael event object.

It works great and keeps my code well organized.  Enjoy!

March Madness

I just created a random bracket generator.

http://freerandombracket.com

Each time you load the page a new random bracket comes up.

Each matchup is decided by a very simple weighted algorithm:

Team Score = sqrt(Team Seed) * <random value between 0 and 100>

For each matchup, the score is calculated for both teams.  Whichever team has the lowest score moves on to the next round.

The page should be printable.

Enjoy!

EDIT:  The above calculation was a little too loose.  It meant that 1 out of 4 matchups between 1 seeds and 16 seeds would be an upset.  So I changed it to the following:

Team Score = (30 + Seed^2)*<random value>

This led to a little more realistic match ups, where 9 times out of 10, a 1 seed would beat a 16 seed, a 1 seed would beat a 2 seed 10% more often than an upset, and an 8 seed would win against a 9 seed 20% more often.

I'm still playing with it, but as of right now that is the algorithm being used.

Manage My MiFi

A few months ago I made a post about a mobile broadband provider called Truconnect.  They provide mobile access without a set plan or contract.  I know the founders personally and after some discussion I proposed an interesting mobile app.  I present Manage My MiFi:

One of the big benefits of using Truconnect is only paying for the data I use (see previous post on why a mobile access without a plan is a good thing).  Since keeping track of that data usage is crucial,  I developed an application for Android and iOS that monitors data usage of a connected MiFi device.  I also built a Mac OS X widget.  It works well and I use it all the time myself.

It works with all MiFi devices (made by Novatel) on all major carriers.  If anyone is interested in Truconnect service, download the free app to learn more (There will occasionally be a promo code in the app for new customers to use, so check before signing up!).


Technical notes:  (for those interested)

The app was a blast to build.  The app concept was conceived just before Christmas and we wanted to launch it in time for CES.  That left about a week to develop the application and submit it to the respective app stores.  I decided to build on the PhoneGap framework, as that would allow cross platform support with minimal effort.  This also made it easy to build a Mac OS X widget.  A Windows Gadget shouldn't be too hard either (let me know in the comments if there's an interest in it).  I actually did a good portion of the development in the car driving home from a skiing trip to Mammoth Mountain.  Surprisingly enough, we got the app finished and approved in time for CES.  It was displayed at the TruConnect booth and was featured in a press release.  I went down for the last day and it was pretty cool to see something I built on display.  Now I want to see people using it!

Space Invaders - 2 Player

Last semester I took a course on embedded systems.  The class revolved around building the classic Space Invaders game.  We built the game on the Virtex-II Development Platform which involves an FPGA with a PowerPC processor.  The combination of a "fast enough" processor and an FPGA meant we could do some pretty cool things.  We developed some custom hardware to act as a DMA audio controller, for example.  One part of the course is a creative assignment, where we were supposed to create a substantial addition to the game.  We decided to make it a two player game, but wanted to do a little more hardware development.  So we picked up two RF transceivers and added wireless communication.  Here's the final product:


Space Invaders - 2 Player from Peter Hamilton on Vimeo.

As you can see, the graphics are pretty good (the video skews them a bit) and the gameplay is very smooth.  My friend Jeremy worked hard to make it an exact copy.  As far as multiplayer goes, the video explains the additional rules for 2 players.  Every time a player kills an alien, there is a 30% chance it reappears on the other persons screen as a yellow alien.  When a player shoots a spaceship, a random number of aliens (between 1 and 7), appear on the other players screen.  The game state is synced over the wireless connection 10 times a second.

This turned out to be a blast.  It was fun to build and pretty fun to play.  It added a very fun competitive element to an a classic game.  We actually left it running in the lab for a few weeks and students in the class would frequently challenge each other.

 Overall it was a pretty cool project.

Thanks to Jeremy Mickelson and Jared Pilcher, my lab partners.

Edit: Hacker news comments

Smuggling data in pointers

While reading up on The ABA Problem I came across a fantastic hack.  The ABA problem, in a nutshell, results from the inability to atomically access both a pointer and a "marked" bit at the same time (read the wikipedia page).  One fun, but very hackish solution is to "smuggle" data in a pointer.  Example:

#include "stdio.h"
void * smuggle(void * ptr, int value){
  return (void *)( (long long)ptr | (value & 3) );
}

int recoverData(void * ptr){
  return (long long)ptr & 3;
}

void * recoverPointer(void * ptr){
  return (void *)( (long long)ptr & (~3) );
}


int main(){
  int a = 90;
  int b = 2;
  int * c = &a;
  void * d = smuggle((void *)c, b);
  printf("The value of a is %d\n", *( (int *) recoverPointer(d) ) );
  printf("The value of b is %d\n", recoverData(d) );
}

The above code outputs:

The value of a is 90
The value of b is 2

So what happened? On a Unix system like OS X or Linux (as well as almost every operating system), memory is aligned to specific addresses. Since a pointer is nothing more than an address, the alignment has an interesting effect on the value of the pointer.

If I run the above code, the integer a has an address of 0x68537518.  This isn't set in stone, and will be different each time I run the program.  If we look at the addresses in binary it looks like this:

0110 1000 0101 0011 0111 0101 0001 1000

Run the code again and we get 0x66601518:

0110 0110 0110 0000 0001 0101 0001 1000

If we look at the address of integer b in the same run we get 0x66601514:

0110 0110 0110 0000 0001 0101 0001 0100

There is a lot of similarity in those addresses, but I want to point out the last two binary bits.  In all addresses, they are zero.  This is because of alignment.  The compiler will align the bits so they always start on a value that is a multiple of four, since integers are 4 bytes (32 bits) long.  This means all integer addresses will always end in 00.

So the smuggle method above takes advantage of that fact.  Since we know the values are always going to be 00, we can replace them with an arbitrary value between 0 and 3.  When we want to retrieve the smuggled data, we only look at the lowest two bits.  When we want to dereference the pointer, we zero those bits out and voila, we have our original address.

In almost all applications, this would be a pretty useless trick, but in the case of the ABA problem, one can provide a couple of flags in addition to the pointer to better describe the value it is pointing to.  This allows instructions such as compare-and-set to atomically access both pieces of data.

Pretty cool little trick, eh?

EDIT:  After a number of comments condemning this practice, I wanted to state that, for the record, you don't want to do this.  There are some corner cases where this is very useful, but generally should be avoided.