Unfortunately, there isn’t a ton of docs to help out but there are plenty of examples within Solr’s source to help.

That being said, here are a few things I found out while working with it.

Because of the way the setUp method worked, I needed to basically duplicate much of its functionality instead of calling super.setUp(). By default, the setUp method will create the data directory for Solr in java.io.tmpdir (generally /tmp on Unix systems) and then the name of the class plus a timestamp. This was a problem for us because it meant that the index would be created for each test in a new directory.

I realize the need for having atomic data for unit tests but I viewed these Solr tests more as integration tests than true unit tests. They were going thru the entire system as opposed to focusing on just one class or section.

To create the index, we were hooking pieces up to our current indexing pipeline, a very nice plug-in system we developed to go through various stations to either clean data or retrieve more of it. Thankfully only a few places actually interacted with Solr so I was able to mock that communication out and just use the data collected and give it to the adoc / update methods.

Because the pipeline wasn’t instantaneous, I wanted to reuse the indexes as much as possible. I figured a good middle ground for this would be for each test class to have its own index and all it to give the indexing pipeline information about what data it wanted to index. That index would stay until a physical directory was deleted and then it would be recreated with updated data.

So I basically had to copy much of the existing setUp method and create the data directory with the test class name but no timestamp as well as make the tearDown method a no-op.

With all of this done, I now have a class which any developer can extend which hopefully will increase our test coverage.

On the side, I’ve been working on a new aggregator, Aggir, which allows me to test various things. I started off using SQLite and Sequel for storage, put Solr behind the scenes for search and added a very simple Web UI using Sinatra and HAML. Yeah, I think I pretty much used all the necessary hot projects. It was fun to build and it works pretty well right now.

I have more to do on the Solr front since I’m just using the defaults for relevance searching. I’d like to dig more into the Solr internals for additional query parsing and classification at index time. It’s some of the stuff I’ve been doing at work but wanted to use a different type of data set.

Of course, now that I had things somewhat stable, I decided to blow it all up and try something new. That something new is Redis using Ezra’s client library.

I started down the path of updating everything, ripping out the database storage to use Redis instead. So far so good, I have the start of this on a branch. One issue I found though was testing my code. It was simple with Sequel since I could create a different database without any worry of overwriting real data. With Redis, I can easily delete keys in between tests but with the keys were the same that a real update would use so non-test data would be deleted.

I think I’ve come up with a solution that at least is working for me. I’ve made each key combine a prefix with other data. The prefixes are defined as class variables. I only set them in the library code if they haven’t already been defined elsewhere. In my test code, I set them with an additional test-specific prefix so that I can easily delete all of the testing keys by using the keys(‘test_*’) method. This will allow me to walk thru all of the test keys created during a test and delete them before running the next test. This mirrors what is done with the database.

I’m now able to test on the same instance that I’ve loaded with posts from various blogs. I have more to say about the mindset change from a relational db to key-value storage but I wanted to get this post out.

What this means for managing projects—including any project that relies on the Internet to deliver its value proposition—is simple: The longer your project timeline, the greater the risk that what you deliver will not be what you or your customers need when you deliver it. Not only are longer-term projects more likely to fail due to changes in requirements or conditions during the project, they’re more expensive. This increases the cost of failure. And because we can only do a few of them in a year, the impact of any one failure is huge.

A very large problem is that everything needs to be setup with production paths or it is almost impossible to get up-and-running. I’ve been spoiled by how Rails handles the environments. Being able to separate what is for local development, what is QA testing and what is for production is an amazing way of allowing developers to get involved quickly and easily.

If you don’t have that, developers will flail around, searching thru config files or trying to follow stack traces, hoping some information can be gleaned from error messages. It isn’t easy and very frustrating and generally causes someone to lose all interest in future development.

A few I’ve seen so far:
• iPhone Application Example
• Inside the iPhone SDK
• Using URL schemes in Apps

I’m sure there will be more but this is a great start… Now if I could just make time to upgrade to Leopard.

• The Las Vegas Sun’s weather page. Extremely clean design with lots of data but easy to read and understand unlike ours.
• EveryBlock is now in LA… Watch out.
• Simple Update Protocol looks like a mix between a cache and sending out Pub-Sub notifications.
• The Onion has invaded Chicago.
• The Blueberry Mint Lemonade from Lemonade is absolutely amazing.
• LED Football for the iPhone.
• Fumblerooski is a perfect compliment to the start of college football.
• Looking forward to trying the Nickel Diner

I’m trying to get CouchDB up and running. Unfortunately, I’m getting no love even after building Spidermonkey. Here’s what I’m getting:

a21772:~/apache-couchdb-0.8.0-incubating jlucas$ sudo couchdbApache CouchDB 0.8.0-incubating (LogLevel=info)
Apache CouchDB is starting.

dyld: NSLinkModule() error
dyld: Library not loaded: Darwin_DBG.OBJ/libjs.dylib
Referenced from: /usr/local/lib/couchdb/erlang/lib/couch-0.8.0-incubating/priv/lib/couch_erl_driver.so
Reason: image not found
Trace/BPT trap

I wish I could get it going as there are some interesting things I want to experiment with.

What you won’t see here are sites or feeds that do not have any unread items. In fact, Brent has taken measures to ensure that on the mobile version of NetNewsWire, the user will only see what’s important to him. Many times during our interview, Brent mentioned that he was developing this application from the perspective of the individual who had only five minutes between tasks to take a quick look at his feeds. To that end, items you’ve read will disappear from the phone in 12 hours or so, keeping what you see on your iPhone to the bare minimum of important items.