technology


The problem with the flare prefuse library is that flare is a commonly used word - so searching for examples on google results in a lot of matches other uses of the word flare. A simple solution is to use site explorer to find links to the flare prefuse website - this naturally disambiguates the meaning.

I found this link confirming to that flare is the best tool to be using  for now

"Yes, but how heavy is Axiis? Have you pushed the limits? I used Ravis Birdeye for our initial efforts for my network mapping app http://www.flairjax.com/?p=72, but the mxml components are way to heavy. Had to go to flare."

  I found this  classification of different types of visualisations

  • data visualisation - numbers in a graph, pie chart, barchart, scatterplot (quantitative data)
  • information visualisation - interactive visual reprentation of information in order to improve cognition
  • concept visualisation - mindmaps and other representations of concepts (qualitative data)
  • strategy visualisation - company strategy - (I dont understand how this in a classification similar to the others listed here)
  • metaphor visualisation - use a metaphores such as tree or landmass to represent something completely different
  • compound visualisation - something that uses a fixture of the above techniques

I think a better classification of a visualisation would be

  1. Is the visualisation user interactive?
  2. Does it represent a moment in time or a change over time  (does it have a time dimension)
  3. Does it contain quantitative data (numbers)
  4. Does it contain qualitative data (opinions of concepts - based on subjective or intangible ideas such as "warmth" or "friendly")
  5. Number of physical dimensions - 1D, 2D, 3D
  6. What visual que's  are used to represent information - color, line patterns, fill area, animation

more to be added to this classification list over time

I decided to clean up the first animation that I had learned from the actionscript animation book. It can also be viewed here.

In the process I made some mistakes and learned a few things:

  1. I extended the object model to be more a mathematical graph based model - with Nodes and Edges - and I tried to abstract out the concept of forces
  2. I made the mistake of trying to iterate over the springs instead of the nodes - this makes life more complex as the forces are applied to nodes - it meant that I was accidentally applying gravity to the node once per spring attached to the node - conclusion: you most iterate over the nodes and then for each node - each force applied to that node
  3. I made the mistake of trying to apply all calculations and rewdrawing the spring lines in the same loop. This does not work because particles later on in the calculation loop will change position. Conclusion is that you need to calculate all of the nodes new positions before having a separate loop with redraws the spring lines
  4. I created the concept of MoorPoint nodes - nodes which do not move in response to forces - only the user's mouse drag and drop actions

The code can be downloaded from here.

After doing all this work - I found that the system was unstable - in terms of a small change in spring value or gravity could result in the nodes disappearing off stage. You can add damping or change the values but I decided to look at some libraries. I downloaded this as3 visualisation library - which is fantastic. They have done a great job of creating a great object orientated model that can be applied. For example they

  • have nodes and edges
  • have separated out the spring, attraction and gravitational forces into implementation of an iforce interface
  • the forces are applied to particles - ie. not to nodes - so the concepts of particle forces and a node-edge graph are kept separate
  • they have rapid node and edge iterators
  • they have the concepts of a fix() attribute that prevents the node from moving when subjected to forces - similar to a MoorPoint
  • node and edge meta data is catered for

Anyone who is working with graphs (node - edges) in no matter what language should look at the flare visualisation library

web graphCame across Barrett Lyon and the opte project - seems kind of interesting

especially the techniques of creating large graphs of the webs connectivity

"Lyon's work tracking Russian Denial of service attack extortion groups has been featured around the globe[4]. He provided details and helped coordinate with multinational law enforcement groups which resulted in the capture of Ivan Maksakov, Alexander Petrov, and Denis Stepanov [5]. The three men were at the heart of an extortion ring which was extorting money from banks, Internet casinos, and other web based businesses. Reported damages caused by Maksakov, Petrov, and Stenanov range in the the tens of millions of dollars. On October 8th 2007, Maksakov, Petrov, Stenanov were found guilty and sentenced to eight years in prison in the Russian Federation with a 100,000 ruble penalty[6]."

  Whist creating my first animation - I was trying to place vertical label text on the sliders. The text was not appearing - there are numerous forumn posts on this subject. This was because I was not using embedded fonts  I managed to get embedded fonts working as follows

[Embed(systemFont='Arial', fontName='spArial', mimeType='application/x-font')]
public static var ArialFont:Class;


var textField: TextField = new TextField();
textField.text = "hello";
textField.selectable = false;
textField.autoSize = TextFieldAutoSize.LEFT;
textField.setTextFormat(new TextFormat("spArial", 12, 0));
textField.embedFonts = true;
square.addChild(textField);

The problem with embedding fonts is that they increase the size of the swf by 100k which is a lot when the origional file was 8k. The following technique renders the text as a bitmap and this can then be rotated:


tmpTextField.selectable = false;
tmpTextField.autoSize = TextFieldAutoSize.LEFT;
tmpTextField.text = _label;
tmpTextField.background = false;


// http://www.actionscript.org/forums/showthread.php3?t=150918
var myBitmapData:BitmapData = new BitmapData(80, 20, true, 0x00000000);


myBitmapData.draw(tmpTextField);
_labelBitmapField = new Bitmap(myBitmapData);


_labelBitmapField.smoothing = true;
_labelBitmapField.rotation = -90;
_labelBitmapField.x = 0;
_labelBitmapField.y = _height;
addChild(_labelBitmapField);

  You can try out my with my first animation here

Several months ago I bought Actionscript 3.0 Animation - Over Christmas I have had some time to play with some ideas. I am interested in visualisation tools - especially things that help technical people understand better what is happening in a complex environment. eg. visualisation of code execution or visualisation of business strategy.

Firstly some key summary points of what I learned from the book:

Sprites are the base class for all interactive elements on the visible screen which is called a Stage. (Actually all interactive elements are derived from DisplayObjectContainer but for the purposes of these animations we work mainly with sprites). The addChild(aSprite) method is what makes the sprite visible on the Stage. You can use removeChild(aSprite) to remove it from the stage. The sprite is not destroyed - just hiding in the wings and not visible to the user. Adding the sprite back onto the stage is a process known as reparenting.

The aSprite.graphics. draw api allow you draw lines and circles associated with that Sprite.

Changing the value of aSprite.rotation property causes the sprite to rotate that value in degrees. All lines drawn by the draw api associated with aSprite will rotate with it automatically.


aSprite.x
aSprite.y

properties values change the sprite position on the stage.

mouse events occur when you are over a sprite but are always available to the stage. ie. if you move over a ball spite and you can have listeners on the stage and the ball, both receiving events


stage.focus = aSprite;

allows you to capture keyboard events on a sprite instead of only on the stage

aSprite.mouseX aSprite.mouseY values are relative to the position of the sprite on the stage.

My first question is : Given that each sprite created has overhead, when do you create something as a seperate sprite?

I think the answer is:

  • When you want to be able to capture events associated with it seperately - for example the handle of a interactive slider is created as sprite
  • When you want to be able to move the object seperately - by simply changing the .x and .y properties
  • When you you want to be able to rotate the object and have all graphics associated with the sprite rotate autmatically
  • When you want to be able to show ( addChild() ) or remove the object (removeChild()) from the stage seperately

so a tree could be

  • single tree sprite
  • part of a forest sprite
  • could have branches as sprites

all depending on what you want to happen in the animation.

Flash's coordinate system is a little messed up - they have x,y position 0,0 in the top left corner and angles (such as aSprite.rotation) run counteclockwise.

to find the angle between to points you can simply use


dx = mouseX - sprite.x;
dy = mouseY - sprite.y;
angleInRadians = Math.atan2( dy, dx );
sprite.rotation = angleInRadians * 180 / Math.PI;

Note: All of the trig functions expect and return radians but sprite.rotation requires 'angle in degrees'.

you can always get the distance between two points using pythagoras theorem

dx = x2 - x1;
dy = y2 - y1;
distance = Math.sqrt( (dx*dx) + (dy*dy) );

Side Note: Adding and removing event listeners in flash is lightweight - so its normal practice to call removeEventListener() when you are in a state where you want to ignore events - rather than have a ignoreEventX flag

In order to perform animation - we may have a set of event listeners for keyboard and mouse events and most importantly:


addEventListener( Event.ENTER_FRAME, onEnterFrame );

listener which is called at the framerate of the flashplayer

inside the onEnterFrame() method - the following code often seems to appear:

  • for each seperate sprite on the stage,
  • take each of the physical forces that are being applied to that sprite broken down as x and y components (use cos and sin to do this)
  • sum these forces and calculate the sprites new position
  • check if the sprite has now gone beyond the stage boundaries
  • for each sprite that has not been already checked - check for a collision
  • if a collision has occured apply interaction logic - could be collision mechanics or attaching the two sprites.

to convert any angle and force magnitude into its x and y components use


vx = Math.cos(angleInRadians) * speedInPixelsPerFrame;
vy = Math.sin( angleInRadians ) * speedInPixelsPerFrame;

We have to run a lot of calculations for each sprite on each frame - so it is common to simply cache the values of xy and vy and work with these directly. Also because it is simply about creating a realistic simulation and speed is actually in pixels per frame - constants like gravity force of G = 9.82 m/s2are simply not used - the developer experiments with the variable constants until a realistic animation is created.

Forces are simply "rate of change of velocity" - so an accelerating spaceship might be modeled as


vx += ax;
vy += ay;

ie. on each onEnterFrame() call - the x and y velocity vectors increase by a value of ax and ay. On the same onEnterFrame() call, the new position of the sprite is then calculated as

sprite.x += vx;
sprite.y += vy;

Gravity on earth is a constant - the distance component is ignored as the mass of the particle vs mass of earth are significantly different - hence we have a y acceleration of:


vy += gravityConstantValueChosenByDeveloper;

Friction force increases the faster something is moving so we use * in the formula:


vx *= frictionValueChosenByDeveloper;
vy *= frictionValueChosenByDeveloper;

Easing occurs when a something moves slower as it gets closer to its target - eg. a person runs up to a wall - The formula therefore calculates the distance between sprite and target


vx = (targetX - sprite.x) * easingValueChosenByDeveloper;
vy = (targetY - sprite.y) * easingValueChosenByDeveloper;

A spring pulls harder the further apart the sprite and the target are. ie. the formula incorporates the distance between the sprite and he target


ax = (targetX - sprite.x) * springStrength;
ay = (targetY - sprite.y) * springStrength;

Offsetting a target: When working with springs or easing its important to be able to calculate the actual target x y cooridates as they may not be the center of the sprite - for example: they could be the nose of the spaceship or end of a bat. In order to do this we use one of our standard trig formulae:


dx = sprite.x - knownX;
dy = sprite.y - knownY;
angleInRadians = Math.atan2( dy, dx );
actualTargetX = knownX + Math.cos( angleInRadians ) * distanceInAStraightLine;
actualTargetY = knownY + Math.cos( angleInRadians ) * distanceInAStraightLine;

Collision detection

basically you have decide if you want to use:

  • flash's built in hit sprite1.hitTestObject( sprite2) - which assumes that the objects are rectangles
  • a distance based collision detection calculation - which favours circular objects

The distance based collision detection calculation formula is simple


dx = sprite2.x - sprite1.x;
dy = sprite2.y - sprite1.y;
dist = Math.sqrt( dx*dx + dy*dy ); // pythagoras theorem
if ( dist > (sprite2.radius + sprite1.radius) ){
trace( "hit");
}

The book gives a simple trick to iterating over each of the sprites only once - this way if object A is not colliding with object B then we dont need to check if B is colliding with A

If you havent worked with java in a while - I think you will find the following process interesting on catching up on where things have moved.

Changes in Java
1) First there is a list of all the enhancements to java over time

Ant and Maven
2) Ant is still great for building a one off custom java app but custom ant scripts for each project have now been replaced with the Maven pom.xml files - which uses "convention over configuration" to standardise the way code is built, packages, deployed etc. This is useful for large orgs of java developers.

Learning to install and work with maven is very quick - 1 day job
Simply go though all the tutorials in the maven book. Eventually you reach the point where there is:

    a single top level pom which "verlocks"(version locks) the versions of the dependencies
    seperate pom files per sub project which simply reference a dependency (no version number is supplied)

Jetty vs Tomcat
3) jetty seems has overtaken tomcat

"

  • requests/responses are sent over a few connections without any significant idle time.
  • Jetty tends to have better scalability when there are many connections with significant idle time, as is the situation for most web sites. Jetty's small memory footprint and advance NIO usage allows a larger number of users per unit of available memory. Also the smaller footprint means that less memory and CPU cache is consumed by the servlet container and more cache is available to speed the execution of non-trivial applications.
  • Jetty also has better performance with regards to serving static content, as Jetty is able to use advance memory mapped file buffers combined with NIO gather writes to instruct the operating system to send file content at maximum DMA speed without entering user memory space or the JVM."

and can jetty can be run from within Maven using

mvn jetty:run

If you see this error message:

"The plugin 'org.apache.maven.plugins:maven-jetty-plugin' does not exist or no valid version could be found"

you have to make certain you are in the correct pom directory - it needs to be the pom one that creates the war

4) Even though java now has annotations - there is an insane amount of xml situps (lots of xml configuration files)

The archi project allows for generation of hello world style code for various frameworks.


[pbirnie@~/java_dev/maven_tut>java -jar archy-0.1.jar
Enter the archetype id [struts2-archetype-starter], or type 'list': appfuse-basic-spring

I completed some IT due diligence inspection of a company for a VC.

Decided to write this post to remind myself about what it important in the process.

  • choose a code name for the project
  • sign the NDA and ensure that you have professional indemnity insurance
  • obtain all technical documentation in electronic form asap
  • obtain company org chart and a basic investor presentation
  • test the existing product
  • https://siteexplorer.search.yahoo.com/ is great for performing a CSI on company history
  • check for IP ownership, SLAs and who is permanent/contract - employment contracts to check IP ownership.
  • preferably obtain all documents in an email (that way you have a record of what was sent to you)
  • setup interviews with technical staff, write up summaries immediatly after the meeting - use my standard questionare
  • write report and follow up with the client

 I previously blogged about Intellij and how to use it to remote debug flex applications.

Not I am changing some of the IDEs settings

Setting utf-8 encoding of files

Now I need to be able to add some other swc files to the path

File->Project Structure

select your Flex module

select the Dependencies tab (A) and click "Add" - then navigate to the additional swc files

After finding this blog entry,  I increase the amount of memory available to the Intellij application

/Applications/IntelliJ IDEA 9.0M1.app/Contents/Info.plist file under your Mac OS X. You have to change the VMOptions key:

VMOptions
-Xms64m -Xmx512m -XX:MaxPermSize=256m -Xbootclasspath/a:../lib/boot.jar -ea

This is a useful how to on setting up a flex unit project

After finding this blog entry and this one I figured out how to compile the swf from the command line

mxmlc compiler options are numerous and can be found here


method not implemented mx.core::IFlexModuleFactory/mx.core:IFlexModuleFactory::allowInsecureDomain()

This error was caused by me compiling against flex sdk 3.4.0 and I actually needed flex sdk 3.2.0

Some interesting tools - to avoid and monitor caching of the swf - JonnyCache and this actionscipt code allows you to see the swf compilation date

I have to say - I have never really liked Eclipse.  Its slow, hard to find the setting you want to change and very painful to configure things like debug. When I was working as a java EJB developer, I used to use Intellij - I loved it. So easy and quick to work with it. Intellij is definately faster than Eclipse - and as a result I feel much more productive. I really pleased to se the following in the new version of Intellij

 

I am working on a flex project now and needed a debugger. I tried for a day to get Flex Builder debugging a remote application (It is easy to get FlexBuilder to debug an application that you launch  - but getting it to accept a connection for a remote application, which in my case is running in my ubuntu vm - seems impossible. The best link I found on setting up FlexBuilder debugging are these freely accessable pages in this OReilly book. The command line debugger work fine - and if you are ok with using that or its a specific one off problem with a machine in a specific enviroment - the fdb command line debugger is an option.

But here are some screen grabs of Intellij 9.0 and some of the important features relating to flex and flex debugging

 

Creating a flex project

 

 

 

You need to have downloaded the flex3 sdk from the adobe site and point IntelliJ to it. You can enable Create Sample (5), html wappper(6) and a custom config file - so that you build from the command line (7)

 

 

I you want the HelloWorld example to run -  you need to do this

 

 

 

 Some important things to note in the IntelliJ 9.0 IDE

 

 

 

 

  • A, B: Tools->Flex -> Create Html Wrapper - allows you to create a html wrapper file for the swf you want to run / debug
  • C: Shows you a line break point I managed to create by clicking in the gutter
  • D:Intellij project files associated with this project
  • E: Shows the flex sdk libraries I am using

 

 The steps 1, 2, 3 allow you to create a run/ debug configuration - you will see I have used a url for the launch details of the application.

 

Now pressing the debug button in the IDE  will open your default browser and the simple Hello World flash application runs  and connects back to the IDE debug port.

 

 

 

 

 

If I click on the button in the flash application, the IDE show the line of code where the break point is set (A), a stack track of my current position in the code (B), values of variables currently in scope (C) and buttons that allow me to single step debug through the flex code(D).

 

When working with a new system - There are various techniques I like to use to understand how it works

  • firebug net tab
  • schemaspy
  • any technique that allows me to print the sql fired on each page request

The following is also useful - A  simple mysql dump of the database schema with the database insert statements listed as one per line. You can then use grep on the 'currentdb' to find the values for settings or simply see where the data is being stored

set -x
FILE='/home/pbirnie/backups/BACKUP_'`date '+%Y_%m_%d__%H_%M_%S'`;
/usr/bin/mysqldump -u root --skip-extended-insert master_openx > $FILE
/usr/bin/mysqldump -u root --skip-extended-insert master_openx > currentdb

I used schemaspy and its meta data feature to create the following ER diagram for Kaltura CE database schema


You can see the full schema spy output here

This is the command line I used


sudo java -jar schemaSpy_4.1.1.jar -t mysql -u root   -db master_kaltura -dp ./mysql-connector-java-5.0.8/mysql-connector-java-5.0.8-bin.jar -host localhost -hq -o /var/www/kaltura_schema_dump  -meta kaltura_schema_meta.xml

here is the kaltura_schema_meta.xml file which contains the relationships between the database entities

only things you can do with data are:

* make more copies of it (eg. master slave replication)
* process data to make more data / summary (eg. monthly weekly rollup tables)
* create different indexes that allow rapid access (eg. index on a, b, c columns, reverse index on all words in a document for free text search )

so for document search  - lets say you have 3 distinct documents A,B,C
and lets assume that these 3 documents sit amongst 300 million documents
and with size of the document data that you are storing - you can only fit 100 million documents per machine (so that the index can fit in memory and the search time is quite short)

so you have 3 machines

FE

A
B
C

and you have a frontend that federates - ie. asks each of the A, B, C machines for hits on the word "dog" and then works out which are the best results to present to the user (based on rank).

if you want to add another 100 million documents - you can just add another row - so you have

A
B
C
D
ie. you can grow with data volume - by adding rows

but if you want to improve performance (ie. reduce latency in the response time) - you can add columns and a round robin load balancing on the front.
ie

AA AA
BB BB
CC CC
DD DD

now lets say you want to run active-active in 2 colos - (2 columns in west coast colo and 2 columns in east coast colo) you could upgrade the index while directing the traffic to the other column in the colo - or if you are using amazon EC could simply rent a new column of servers - create the search index on them and discard the old column  when   you are ready

I am still trying to work out how companies manage to  stream video economically. This trace route mashup up that I found on programmableweb.com is great though

 

3 types of people doing developing open source

  1. young techi who works for a company - he doesnt have to worry about how to make money from open source - He is kewl 'cos he is doing open source
  2. hardcore techi who in his spare time contributes to a open source project - he does it because he loves it
  3. techi who is building open source products as a distribution channel - and wondering if it is possible to make any money from open source

     I went to the cloud computing / gaming conference yesterday

1) the organisers where shocked by the turn out - 150 attendees -  they where only expecting 50

2) lots of gambling / spreadbetting people there

3) Most / all products talked about where focused to the high throughput requirements of gaming platforms. Most of them are java based - focus is on storing data in memory and persisting to db asynchronously (see gigaspaces)

4) Met various people - including Simon Wardley at Canonical (ubuntu) - his talk was amazing - will send you link to video when its up

"Cloud computing is a consequence of economic and technical conditions that have combined to cause a disruptive change in IT towards a service based economy."

5) linked to about 6 people on linkedin  - one person might be looking for video ads work and other is a Serbian gaming developer consultant

6) introduced yet another person to Roy at shinyads

7)

  •     EC computing allows little guys to take on big guys.
  •     EC computing - even for 50 machines is cheaper than having an IT manager in your company.
  •     Important to manage expectations with business when saying we are moving to cloud - because its all new
  •     Demand to read the full SLA when moving to cloud computing
  •     Many people are concerned about actual failover that amazon has (not enough colos)
  •     Looks like amazon EC's api will become the standard api - Ubuntu is pushing a open source version of EC called Eucalyptus
  •     Be careful what prepackaged instances you will use on EC - some of them have alpha versions of code (eg. java7 and tomcat)
  •     Some gaming companies store their data in Gibraltar and use amazon EC in Ireland for processing (some latency concerns).
  •     For most companies where the data is stored has legal issues - eg. US laws vs EU laws
  •     Someone quoted 4 months at a mobile operator project to get hardware requisitioned (load testing, product purchase codes etc) vs could have used EC in 10 min - the service wasn't that popular when it went live. One person quoted cost  $90 on EC vs $700k(using internal hardware)
  • Some gambling companies use the cloud to absorb peak load - for example just before a horse race and use existing hardware for normal day-to-day operations

8) interesting Akamai will not accept egaming companies - Some companies are using Amazon Cloudfront (Amazons CDN)

9) RabbitMQ - open source async message broker built in erlang - has one user soocial.com ... - an address book syncing tool - how origional

10) Chris Anderson - the guy who wrote the long tail book (and invented the term) - has a new book out now called free (I blogged about this book 1 year ago)

http://www.wired.com/techbiz/it/magazine/16-03/ff_free

"Over the past decade, however, a different sort of free has emerged. The new model is based not on cross-subsidies — the shifting of costs from one product to another — but on the fact that the cost of products themselves is falling fast. It's as if the price of steel had dropped so close to zero that King Gillette could give away both razor and blade, and make his money on something else entirely. (Shaving cream?)"

   7 days to go until I fly to California for OSCON. Apart for learning lots, I hope to make some good business contacts there.

I have printed 3 tshirts - I have the slogan "VideoAds open sourced..." printed high on the back of the tshirt - I used this service - I really like the in browser tshirt design tool - we will see what they look like

I printed 2 sets of business cards - one set to promote bouncingminds VideoAds solution and one set to promote pure consulting work though Zulutime Solutions. I used www.fastcards.co.uk - their in browser business card design tool is terrible - I must have looked at 10 online business card printing companies - moo.com seems to be the only decent one. I couldnt use moo because they require at least 5 days to print business cards. Perhaps just having a usable web interface is enough to give a business a significant advantage over compeditors.

I designed my Zulutime company  logo - perhaps I should have paid someone to do it ... but the logo has lots of hidden meanings for me which I was thinking about . On thurday I attended a talk by Simon Wardley from Canonical at skillsmatter. I am still waiting for the video of the talk to go up online (so I can place a link to it - but it was similar to this one) - A lot of the talk was about how technology goes though cycles of invention->bespoke->product->service on a sigmoid curve (hence the Sigmoid in the logo).

The green & gold is South African colors, the egg for invention and also the symbol looks a little like a platapus - the green arrow is for time. The little baige symbols are  sails, shields or ships showing the movement up the curve.

OMG - I am still in shock trying to work out what google wave means. Is it the end of facebook, twitter, wikipedia and webpages - or are these sites just skins for the content - like email client wars that happened many years ago

How will sites keep control of content generated on their network? And more importantly make money from the ads next to them? Rss feeds where great as teasers to drive traffic to the site - but now any developer can create an app that sucks content from facebook - so the warriors attacking facebook are the users - Google just created the platform.

I like google docs but it doesnt have all the features I want - so I prefer to use a purpose build tool on my machine - I assume someone will soon create a filesystem that will proxy to wave when I save my document in MS word or on Mac Pages.

also not featured in the demo is the difference in context between methods of communications - for example: I  like to send emails for things that dont have to be done right now or can be publicly shared later as a record of events - and messenger is more for non-official conversations.

There as to be a massive market for selling wave style app into corporates - plus integration work. The fact that the corporate could host the wave app internally solves company paranoid concerns about data hosted externally plus it has built in access control on each entry. Internal corporate search tools represents a large market. Wave could do with a plugin that searches for questions in users conversations and matches it against known corporate approved answers.

  OpenX ad platform supports ad ops staff configuring limitations in the number of times a user sees an ad in a defined session duration.

I used firebug to inspect the cookies passed on each page request and xdebug on the server. As far I can tell the impression counters use two http requests to detect fraudulent impressions. So the 2 requests set, check and clear values in the cookies - this creates a state machine that will not bump the impression counter unless a ad request is followed by and ad impression (lg.php)

The first request is the request for the ad for a particular zone (and receives cookies as part of this response)
The second request is the impression request - which asks for a 1x1 image (and receives cookies as part of this response)

 
request for ad for image banner ad
AID=1da02e891d41e726c1089be4e17d5ca8;
OABLOCK=17.1245857291;
sessionID=965ccec6008179bb5b31bea3c2cfe172
; PHPSESSID=e8629714b631a1a0e0dab302361485d5;
OASCAP=17.5; <--- the session (S) level counter for banner 17 is 5
_OASCAP[17]=1; <--- this the marker that the previous impression occured (it is set by the previous lg.php call)
_OABLOCK[17]=1245857300
 
response with banner ad
 
_OABLOCK[17]=deleted; expires=Tue, 24-Jun-2008 15:29:11 GMT; path=/
%5FOABLOCK%5B17%5D=deleted; expires=Tue, 24-Jun-2008 15:29:11 GMT; path=/
_OASCAP[17]=deleted; expires=Tue, 24-Jun-2008 15:29:11 GMT; path=/
%5FOASCAP%5B17%5D=deleted; expires=Tue, 24-Jun-2008 15:29:11 GMT; path=/
OAID=1da02e891d41e726c1089be4e17d5ca8; expires=Thu, 24-Jun-2010 15:29:12 GMT; path=/
OABLOCK=17.1245857300; expires=Fri, 24-Jul-2009 15:29:12 GMT; path=/
OASCAP=17.6; path=/ <----- bump the counter for video ad 17 to a value of 6 (OpenX can see _OASCAP=17.5 AND _OASCAP[17]=1
 
request to log (lg.php)
OAID=1da02e891d41e726c1089be4e17d5ca8;
OABLOCK=17.1245857300;
sessionID=965ccec6008179bb5b31bea3c2cfe172
PHPSESSID=e8629714b631a1a0e0dab302361485d5
OASCAP=17.6
 
response from logging (lg.php)
OAID=1da02e891d41e726c1089be4e17d5ca8; expires=Thu, 24-Jun-2010 15:29:12 GMT; path=/
_OASCAP[17]=1; path=/ <---- This is marking that the impression occured, for the next request
_OABLOCK[17]=1245857352; expires=Fri, 24-Jul-2009 15:29:12 GMT; path=/

I found a new favorite search engine for travel - http://www.kayak.co.uk/ - its yet another flight / car / hotel search engine - but its interface is clean and fast - very good UX.

One of the things I like about it is its faceted search - ie. On the left pannel there is a slider where  I can specify a range of take off times and a range of landing times. I assume that these changes make a call to faceted search engine like Lucene - but there is no reason they couldnt just be a dynamically built sql query into a mysql database.

In my minds systems like Lucene have the following properties:

  • superior fuzzy text matching capabilities including custom ranking algos, stemming
  • slow to index (slow to update data)
  • good support for clustering (federated search)

Mysql has the following properties

  • nice well know SQL interface
  • suite of well tested and known tools to support replication,  turning etc
  • fast to update index (for rapidly changing data - like product offers)
  • could be used in a clustered / federated mod. ie call 10 instances of mysql - each with their own portion of the data - then merge data and re-rank on the in the fedorator component
  • support for plugin-able backend query engines (which could be a webservice call to Lucene)

Next Page »