Integrating batsd with graphite and statsd

At Mailprotector we recently integrated batsd with our graphite statistics and metrics system. The result is a faster and native interface for querying our metrics data using ruby. This change has made it easy for us to create our own dashboards and reports in our Rails apps.

Since batsd was built with compatibility for statsd, integrating was simple. Using the repeater option for statsd, we are able to continue to send the data to graphite but also repeat it to our batsd system.

Here is an example of the repeater configuration.

backends: [ "./backends/graphite", "./backends/repeater" ],
repeater: [ { host: '1.2.3.4', port: 8125 } ]

Now we have the data in two places. We can also optimize the retention in batsd for our needs there and keep using graphite setup the way it was.

The Batsd Client Gem

We needed a client to query the data from batsd for our dashboards and reports we wanted to build in our internal Rails apps. There is an example ruby client provided in the batsd repository but nothing packaged up and ready to go. So using that example as a starting point, I created a batsd ruby client and bundled it up as a gem.

Installation

It’s a simple gem installation.

gem install batsd

Or you can include it in your Gemfile.

gem "batsd"

Getting Started

You can connect to Batsd by instantiating the Batsd class:

client = Batsd.new

This assumes Batsd was started with a default configuration, and it listening on `localhost`, port `8127`. If you need to connect to a remote server or a different port, pass in the appropriate options:

client = Batsd.new(:host => "10.0.0.1", :port => 8127)

The options and defaults are:

:host => "127.0.0.1"
:port => 8127
:timeout => 2000 #milliseconds
:max_attempts => 2

Now you can grab the list of available keys:

keys = client.available

To pull the stats for a key:

start_timestamp = Time.now - (60*60) # 1 hour ago
end_timestamp = Time.now
stats = client.stats("metric_name", start_timestamp, end_timestamp)

Each stat is returned as a hash with a `:timestamp` and `:value`. To pull only the timestamps or values for a range:

start_timestamp = Time.now - (60*60) # 1 hour ago
end_timestamp = Time.now
values = client.values("metric_name", start_timestamp, end_timestamp)
timestamps = client.timestamps("metric_name", start_timestamp, end_timestamp)

Pull requests are always welcome. Just fork the project.

How to share a list from the Reminders app

One thing that I immediately wanted to do with the Reminders app both in iOS and now on Mountain Lion is create a shared list for me and my wife for things like groceries. It seemed like such an obvious use for this app and iCloud. So I created the list in the iOS app and looked for the share option. To my surprise, there isn’t one. So I assumed this was a limitation on the iOS and opened up the Reminders app in Mountain Lion. It wasn’t a limitation, the “share” button just doesn’t exist.

So after being disappointed, I thought I’d give this one more try by checking the iCloud web app. There isn’t a Reminders app there but the Reminders are nothing more than task lists keep in the calendar.

I was pleased to see the little share icon next to the Reminders lists just like with calendars. Simply click that icon and you’ll be able to enter the email address of another iCloud user you want to share the list with. Once that user approves the invite (from the email they receive after inviting them) then the list will show up in their Reminders app on both iOS and Mountain Lion.

I’m guessing this process will be improved in future updates the Reminders app both on iOS and Mountain Lion. It definitely needs the addition of the share option.

How to fix git after upgrading to Mountain Lion

During the process of upgrading to OS X Mountain Lion (10.8), it seems that git was removed so I was unable to run commands from Terminal.

$ git -v
-bash: git: command not found

This was a big problem for me since I typically run a git command within the first 15 minutes of sitting down at my computer.

Fixing this wasn’t as easy as I had originally thought it would be. Previously I installed git using Homebrew. Unfortunately, simply running `brew install git` didn’t do the trick this time. First it threw a warning about having and old version of Xcode (4.3.3) and then it failed to compile.

$ brew install git
Warning: You have Xcode-4.3.3, which is outdated.
Please install Xcode 4.4.
Error: Failed executing: make prefix=/usr/local/Cellar/git/1.7.11.3 CC=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang CFLAGS=-Os\ -w\ -pipe\ -march=native\ -Qunused-arguments\ -mmacosx-version-min=10.8 LDFLAGS=-L/usr/local/lib install (git.rb:49)

I figured that the warning and the failure to compile were probably related so I started by updating to Xcode 4.4. Since Xcode is distributed through the App Store now, this should be simple right? Wrong. At the time the App Store was still showing version 4.3.3. So I had to download Xcode 4.4 directly from Apple’s developer website. If you aren’t an Apple developer you may be stuck waiting for the App Store to update with 4.4. Update: It looks like the App Store has updated with Xcode 4.4. (Thanks Adam!)

Xcode 4.4 dosen’t install the command line tools automatically. You have open up Xcode, go to Preferences -> Downloads then click install on command line tools. (Thanks to Jon’s tip in the comments)

Now you will be able to install git again using Homebrew:

brew install git

How to get the latest ruby versions for rbenv/ruby-build

If you’ve been using rbenv to manage your rubies for a while, most likely you’ve come across the situation where you need to install a newer build of a ruby version but it isn’t available when you run `rbenv install`. If try to install it and you get something like this:

rbenv install 1.9.3-p125
ruby-build: definition not found: 1.9.3-p125

The key to getting this list updated is updating not rbenv but ruby-build since that is what manages the installation of the rubies. Depending on whether you installed directly by cloning from the github repository or used a package manager like Homebrew (like I did), you’ll need to follow the correct steps for updating.

Updating ruby-build using Homebrew

Updating ruby-build with Homebrew is as simple as running the install command again and forcing it to link the newly built binaries.

brew install ruby-build
brew link -f ruby-build

Updating ruby-build using git

To update ruby-build from the github repository, you’ll simply need to pull the newest version and run the install script.

cd ~/.rbenv/plugins/ruby-build
git pull

That’s it! Now when you run the `rbenv install` command, you’ll have access to the latest builds of ruby.

Twitter loses the hashbangs

When you come to twitter.com, we want you to see content as soon as possible. With hashbang URLs, the browser needs to download an HTML page, download and execute some JavaScript, recognize the hashbang path (which is only visible to the browser), then fetch and render the content for that URL. By removing the need to handle routing on the client, we remove many of these steps and reduce the time it takes for you to find out what’s happening on twitter.com.

Great move. Hashbangs were a hack bad idea from the beginning.

Diet Coda

Another great app from Panic that allows you to create on an iPad. Also, this may be the best name for an app, ever.

FaceTime over 3G

Warnings contained in the latest version of iOS suggest that Apple plans to bring support for 3G wireless data connections to its FaceTime video chat feature.

FaceTime on 3G… no. FaceTime on 4G… yes.

Shortcuts are just a distraction

Sometimes the quest for a silver bullet comes from a good place. Technology is about removing mindless work. Great engineers go through a phase where they try to create the ultimate framework to remove their daily grind, only to realize it’s more work to maintain a gigantic framework.

The difference is shell apps come from the wrong mentality. They start from, “How do we reduce effort?” instead of “How do we deliver the best product?”

Great products require more work. They require commitment, attention to detail, and leaving your comfort zone. Shortcuts are just a distraction.

Great article on developing great apps and the mindset required to do so.

Improving text editing on the iPad

This is a nice approach to improving text selection and editing on the iPad. I agree that the current options are slow and a bit clunky to use. I don’t see Apple going with this exact approach but I do see this as an area they will try to improve with gestures.

← Older posts