Contact

Ryan Cox

Popular Posts

java.util.uuid FAQ
XPath Engine Comparison
Author Tips

 RSS Feed

Jun
11th
Wed
permalink

Greg Kroah Hartman did a techtalk at Google recently. Some interesting stats related to the Linux 2.6.25 kernel development effort:

  • 2399 contributers over last 1.5 years
  • 1/2 of these have done one patch, 1/4 did 2
  • 9.2M Lines of Code
  • 10% year over year lines of code growth
  • ~55% of code in drivers - majority by a small bit, but not all drivers.
  • Change rate of kernel code to driver code is proportional. Translation: the core of OS is changing at a crazy rate.
  • New releases every 2.75 months or so
  • 9000 symbols and 12000 data structures inside the kernel (below the syscall layer)
  • Rate of change is insane
    • 4300 lines added
    • 1800 lines removed
    • 1500 lines modified
    • PER DAY!!!

Comments (View)    
Jun
8th
Sun
permalink

Pythonic API for Tumblr's web API checked in

When my Wordpress blog got hacked, I decided it was time to port over to Tumblr and let them deal with hosting, patching, etc… I put together a python script to import my exiting content via RSS. My original script grew into python-tumblr. Several samples are checked into SVN including one that uses Google Charts to visualize post type frequency.




Comments, feedback, patches welcome…
Comments (View)    
Dec
16th
Sun
permalink

Google Charts Gets Invader'd

Playing with Google Chart API and PyGoogleChart this afternoon after reading the announcement last week.

Invader Chart

A couple things to think about when using the API:

  • You will want to use a wrapper - Take one look at the link that generated the above graphic, and you’ll quickly see that this is not meant for humans. Maybe write your own wrapper around the bits that you are using. Maybe use one like PyGoogleChart.
  • It’s not just for web apps - Charts can be cached for later use using curl or equivilent. NB: IANAL so review the Terms of Service and usage policy and make your own decision.
  • You will have to normalize your data - The chart api docs talk quite a bit about reducing number of elements in your datasets. Another important thing to think about is the range. For instance if I wanted to chart the last few days closing values for the CAC 40 Index which varies from 5,547.21 to 5,750.92. I would have to first normalize the values to something under 4096 for Extended Encoding, under 100 for Text encoding and less than 62 for Simple Encoding. Then the y-axis labels would have to be adjusted to reflect the normalization. Example

It’s pretty clear we’ve only seen the beginning. People outside will find new and unanticipated ways of using the API and Google will continue to crank out new features.

Inspiration for above:

Comments (View)    
Mar
3rd
Sat
permalink

Luke 1.7 is out

Luke 1.7

After a year or so of shut-eye, Luke, the Lucene query tool, is back. New features listed below. Time to dust off some patches I’ve got for an earlier version and send them along for 1.8.

Luke 1.7 features:

  • Added pagination of results, especially useful for very large result sets.
  • Added support for new Field flags. They are now displayed in the Document details, and also can be set on edited documents.
  • Added a function to add new documents to the index.
  • Low-level index operations (such as detecting unused files, index directory cleanup) use the newly exposed Lucene classes instead of duplicating their internals in Luke.
  • A side-effect of the above is the ability to properly cleanup all supported index formats, including the new lockless and single-norm indexes.
  • Added a function to copy the list of top terms to clipboard.
  • Added a function to copy the term vector to clipboard.
  • Added a function to close and/or re-open the current index.
  • In the Documents tab, pressing “First Term” now positions the term enumeration at the first term for the selected field.
  • Added a field vocabulary analysis plugin by Mark Harwood, with some modifications.
  • Overall UI cleanup - improved layout in some places, added graphics instead of ASCII art, etc.

Nicely done Andrzej.

In other Lucene related news, it looks like some interesting things are brewing in the Hadoop project.

Hadoop is a framework for running applications on large clusters built of commodity hardware. The Hadoop framework transparently provides applications both reliability and data motion. Hadoop implements a computational paradigm named Map/Reduce, where the application is divided into many small fragments of work, each of which may be executed or reexecuted on any node in the cluster. In addition, it provides a distributed file system (HDFS) that stores data on the compute nodes, providing very high aggregate bandwidth across the cluster. Both Map/Reduce and the distributed file system are designed so that node failures are automatically handled by the framework.

Comments (View)    
Feb
24th
Sat
permalink

The Definitive ANTLR Reference


As reported elsewhere, the Pragmatic guys have shepherded another great book through their process: The Definitive ANTLR Reference. Though not intended as a how-to, I’m finding it very readable.

Last time I looked at ANTLR, the ANTLRWorks tool ( pictured above ) didn’t exist. It’s got some quirky aspects and rough edges, but it makes exploring and learning by example a reality. Recommended!

Comments (View)    
Jan
1st
Mon
permalink

VIM Plugin for JavascriptLint


I’ve put together a vim compiler plugin to provide quick access to JavaScriptLint output. It will identify file, line and column of warnings and errors generated by jsl.

Requirements:

  • vim 7+
  • jsl in path

Installation:

  • Download vimball - jsl.vba
  • Open vimball - vim jsl.vba
  • Verify contents - :VimballList
    Should see:
    would extract <plugin/jsl.vim>: 1 lines
    would extract <compiler/jsl.vim>: 22 lines
    
  • Install - :so %

Usage:

  • Open file - vim jsl-test.js
  • Run JavaScriptLit - :make
  • View output - :cope

Feedback welcome…

Comments (View)    
Jun
21st
Wed
permalink

Raleigh NFJS 2006


Recently Jay brought his current lineup of speakers to town for the 2006 NFJS Symposium Tour . Lots of great speakers. Packed with nuts and bolts details. It was buzzword compliant, with plenty of talks on AJAX and Ruby. Justin Gehtland’s talks on hibernate were excellent and can be found on codecite . These are polished speakers who have given these sessions many times prior. Dave Thomas could make event the most prosaic topic engaging and entertaining. The rare bit of wit, humor and humanity go a long way in this field.Points that stood out:

Keep Track of Your Ideas

Andy Hunt talked at length about keeping track of the ideas that occur to you throughout the day. Method doesn’t really matter. Some techniques he proposed: spacepen and pad, voice memos on your cell phone, wiki or wiki modes in your editor. I’ve installed potwiki to give it a try. Elsewhere on the web Michael Michalko seems to have a few things to say on this topic.

I like Michalko’s suggestions for developing ideas once captured:

  • What can be SUBSTITUTED? (Who else? What else? Other ingredient? Other process? Other power? Other place? Other approach? Can you change the rules?)
  • What can be COMBINED? (How about a blend, an alloy, an assortment, an ensemble? Combine units? Combine purposes with something else? Combine appeals? Combine ideas?)
  • What can I ADAPT from something else to the idea? (What else is like this? What other idea does this suggest? Does the past offer a parallel? What could I copy? Whom could I emulate?)
  • What can I MAGNIFY? (What can be added? More time? Stronger? Higher? Longer? Extra value? Extra features? Duplicate? Multiply? Exaggerate?)
  • What can I MODIFY or change? (What can be altered? New twist? Change meaning, color, motion, sound, odor, form, shape? What other changes can be made?)
  • Can I PUT the idea TO OTHER USES? (New ways to use as? Other uses if modified? Can you make it do more things? Other extensions? Other spin-off? Other markets?)
  • What can be ELIMINATED? (What to subtract? Smaller? Condensed? Miniature? Lower? Shorter? Lighter? Omit? Streamline? Split up? Understate?)
  • What can be REARRANGED the parts? (What other arrangement might be better? Interchange components? Other pattern? Other layout? Other sequence? Transpose cause and effect? Change pace? Change schedule?)
  • Can it be REVERSED? (Transpose positive and negative? How about opposites? Turn it upside down? Reverse roles? Consider it backwards? What if you did the unexpected?)

Class Oriented Development

During his keynote Dave Thomas took more than a few well-deserved pot-shots at the received wisdom in the software community. He ticked through some of the usual suspects: type safety, methodologies, backward compatibility. Simply stated, his thesis was: if you don’t a have direct and clear understanding of why you are doing something: stop doing it. One point that resonated with me was his characterization of Java and many other object-oriented languages as really class-oriented. We spend our days developing sophisticated class relationships and hierarchies. Is-a. Has-a. While these languages provide anemic support for actually orienting your development around objects; the heart and soul of your application at runtime. Few true hierararchical taxonomies exist in the real world. Classes were invented by stamp-collectors. Why are we trying to put the world into neat little boxes?

This was of course a shameless plug for features like mix-ins that exist in many modern programming languages . Either way this is an idea worth kicking around the block.

Learn By Doing

In his Refactoring your Wetware talk Andy Hunt drew on the example of surgerical residents who follow the pattern of observe, assist and perform. For surgeons there is no substitute for doing. The same is probably true for developers.

Put down that book. Close Firefox. mkdir testapp

Comments (View)    
Jun
13th
Tue
permalink

Vim7 Feature Roundup

I noticed Emacs 22 getting some press lately. So figured it was time to spread a little vim7 love .

Here is a grab bag of features I noticed.

Omni Complete


Completion now comes to vim. <ctl-x><ctl-o> and vim does it’s best to complete the symbol. It’s not perfect. The implementations look like they are mostly regexp’ing around. No knowledge type systems. Based on the contents of $VIM/autoload it looks like there’s out of the box support for: XML, HTML, CSS, C, JavaScript, PHP, Python, Ruby, and SQL. Sorry Java!

Quoted Text Select


New quoted text selection shortcuts while in select mode. With cursor in a string <ctl-q>i” will select all the text within double quotes. If you want the quotes <ctl-q>a” will do the trick.

:vimgrep


Vim now comes out of the box with a grep implementation that works well with existing errorlist functionality.Recursively look for all the FIXME tags in the ruby source tree.:vimgrep /FIXME/j C:/ruby/src/**.rb

View the hits and step through them

:cope

Tabs


Enable this feature and you’ll feel like you walked into a room with 3 switches that control the same light. It looks nice, but when cycling through buffers with :bnext and you can see any buffer in any tab. It’s there if you want it, but I suspect this will largely go unused by most gvim people.To open a new tab:

:tabnew

Spell Check


It’s 2006! I want a jetpack not spell checker!

Nothing too surprising here. Ships with English support. You can grab OpenOffice dictionaries and use the :mkspell to convert it into something vim can cope with. Notice red squiggles under the spelling errors and blue under the punctuation errors.

:setlocal spell spelllang=Fr_fr

Undo Branches


This is an excellent feature with a confusing name. Think of it as restore to any point in time during an edit session. It will save your bacon when you make some edits, undo a few levels, make some more edits and realize you wanted something back from before the undo. New commands support restoring to a text state based on time:

  • g+ / g- This is like u / CTRL-R except it will walk across all changes over time. Good candidates for mapping.
  • :earlier 5m Restore to state 5 minutes ago
  • :later 1h Restore to state 1 hour later
  • :undolist Show leaves in undo / redo tree

:MkVimball

Package and distribute plugins and vim macros into vba files with this new command. When opening vim will recognize the vba files and prompt to extract. Uninstall would be a nice addition.

:GetLatestVimScripts

A very lightweight package manager is now included. Make sure you have a version of wget in your path. Then create a file $VIM/getlatest/GetLatestScripts.dat that looks like:

ScriptID SourceID Filename
————————–

A call to :GetLatestVimScripts will then attempt to update all plugins aware of this new functionality.

It will update the file with version and date information. Note that vim ships with more plugins than this, but GLVS, netrw and vimball are the only ones aware of this new feature.

ScriptID SourceID Filename
————————–
642 5228 :AutoInstall: GetLatestVimScripts.vim
1075 5814 :AutoInstall: netrw.vim
1502 5828 :AutoInstall: vimball.vim

Script Enhancements

Vim’s macro language continues to evolve. Things like this are now possible:

:let num = { 1: ‘one’, 2: ‘two’, 3: ‘three’ }
:echo num[2]

One step closer to emacs convergence?

Comments (View)    
Mar
22nd
Wed
permalink

Music to code by!


The already great somafm just launched space station soma.

Tune in, turn on, space out. Spaced-out ambient and mid-tempo electronica.

Comments (View)    
Dec
6th
Tue
permalink

Aardvark'd: 12 Weeks With Geeks


I finished Joel’s Aardvark’d documentary last night. It captured the experiences of his 4 interns for 12 weeks. Pointing a camera at 4 guys typing for 3 months and editing that into something entertaining is non-trivial. I struggled to find the thesis, but there was plenty to like.

Some observations:

What it is:

A little bit of advertising. Without being cynical, let’s be realistic. I paid $20 for this thing. Now I’m blogging about it. Mission accomplished.

A look through the key-hole at this medium’s potential. Cameras and computers are cheap. There’s more of this type of thing to come and they will build it’s strengths.

A tease for folks interested in the Y Combinator. Paul Graham’s work in Boston is briefly contrasted. Someone send a guy and a camera up there!

What it might be:

A HOWTO for companies starting internships. There more than a few lessons to be learned: find them housing, buy them a TV, let them get some skin in the game the first day by putting together their workspace, give them the same hardware as the rest of the team, give them a spec, etc…

A depiction of the banality that comes along with software development. Long nights where the only thing interesting that happens is a cockroach crawling across the floor. Endless debates in the forums. The office manager / secretary provides us the view of some one looking in from the outside.

What it wasn’t:

A deep look into the journey of the intern; intern as Ubermensch; emerges from long dark 12 week tunnel having re-evaluated prior ideals.

An insightful portrayal of the starts and stops of software development; the highs and lows; peaks and valleys. While some demos crashed and some bugs were found, the contrast between the intoxicating sensation of shipping and sober realities along the way went largely unaddressed.

Comments (View)