The Agile toolbox
Some new businesses go out and raise millions in capital before they do anything else. Not us — we only do what we can afford. Money makes you lazy. It's technical consulting on a shoestring!
If you're on a budget, open source is your best friend. More than this, it's important an open toolbox is less dependent on hardware and less tied to workflows. Better yet, avoiding large technology investments helps us avoid vendor lock-in, and the resulting data lock-in, keeping us more agile. And there are two more important things about open source:
- You know exactly what the software does, because you can read the source code
- You can change what the software does, becase you can change the source code
Anyone who has waited 18 months for a software vendor to fix a bug or add a feature, then 18 more months for their organization to upgrade the software, knows why these are good things.
So what do we use?
In the light of all this, people often ask us what software we use to get our work done.
Hardware Matt is usually on a dual-screen Apple iMac running OS X 10.6, while Evan is on a Samsung Q laptop (with a sweet solid-state drive) running Windows. Our plan, insofar as we have a plan, is to move to Mac Pro as soon as the new ones come out in the next month or two. Pure Linux is tempting, but Macs are just so... nice.
Geoscience interpretation dGB OpendTect, GeoCraft, Quantum GIS (above). The main thing we lack is a log visualization and interpretation tool. Beyond this, we don't use them much yet but Madagascar and GMT are plugged right into OpendTect. For getting started on stratigraphic charts, we use TimeScale Creator.
A quick aside, for context: when I sold Landmark's GeoProbe seismic interpretation tool, back in 2003 or so, the list price was USD140 000 per user, choke, plus USD25k per year in maintenance. GeoProbe is very powerful now (and I have no idea what it costs), but OpendTect is a much better tool that that early edition was. And it's free (as in speech, and as in beer).
Geekery, data mining, analysis Our core tools for data mining are Excel, Spotfire Silver (an amazing but proprietary tool), MATLAB and/or GNU Octave, random Python. We use Gephi for network analysis, FIJI for image analysis, and we have recently discovered VISAT for remote sensing images. All our mobile app development has been in MIT AppInventor so far, but we're playing with the PhoneGap framework in Eclipse too.
Writing and drawing Google Docs for words, Inkscape for vector art and composites, GIMP for rasters, iMovie for video, Adobe InDesign for page layout. And yeah, we use Microsoft Office and OpenOffice.org too — sometimes it's just easier that way. For managing references, Mendeley is another recent discovery — it is 100% awesome. If you only look at one tool in this post, look at this.
Collaboration We collaborate with each other and with clients via Skype, Dropbox, Google+ Hangouts, and various other Google tools (for calendars, etc). We also use wikis (especially SubSurfWiki) for asynchronous collaboration and documentation. As for social media, we try to maintain some presence in Google+, Facebook, and LinkedIn, but our main channel is Twitter.
Web This website is hosted by Squarespace for reliability and reduced maintenance. The MediaWiki instances we maintain (both public and private) are on MediaWiki's open source platform, running on Amazon's Elastic Compute servers for flexibility. An EC2 instance is basically an online Linux box, running Ubuntu and Bitnami's software stack, plus some custom bits and pieces. We are launching another website soon, running WordPress on Amazon EC2. Hover provides our domain names — an awesome Canadian company.
Administrative tools Every business has some business tools. We use Tick to track our time — it's very useful when working on multiple projects, subscontractors, etc. For accounting we recently found Wave, and it is the best thing ever. If you have a small business, please check it out — after headaches with several other products, it's the best bean-counting tool I've ever used.
If you have a geeky geo-toolbox of your own, we'd love to hear about it. What tools, open or proprietary, couldn't you live without?
Matt Hall
Just heard about this awesome list by John Stevenson, a postdoc at Edinburgh University, UK. Thanks to Jon Tennant, aka @protohedgehog, for the tip. Don't know how I missed that...
agile, business, open source, tools in Software 
Reader Comments (13)
I have to get around to creating my own "colophon" but, until then, check out James Fee's personal GIS stack. Probably lots of useful stuff in there for you guys, for generating maps and keeping track of spatial/geographic data.
Interesting post - thanks! I always like discovering new software and new tools, and I'm gonna check out Tick and Wave and Mendeley right after this. I can use the first two in particular right now.
I'm glad to see that at least 50% of Agile is operating on Apple, hopefully it'll be 100% soon (Macs have cool SSDs too Evan). Although, I must say that a full Linux distribution would be more appropriately open (and free!). One question though - do you find the iMac has enough horsepower for the more intense computational stuff? My own computer is starting to sound like it's grinding coffee all day, and I'm going to have to look at an upgrade soon.
Before I read your post I was reading another blog that I wanted to share with you and Evan - it seemed right up your alley in so far as it's about an extraordinarily successful company and their unconventional corporate structure (or lack there of). The company is Valve Software (makers of Half-life, Portal, Team Fortress, Left 4 Dead, Steam, et cetera), and the blog is here.
That post is about 1/3 history lesson, 1/3 business discussion, and 1/3 sales pitch, but I think it's interesting anyway. Michael Abrash is the author, and for those of us who've done some 3D programming in the dark early days of 3D programming, he's kind of a big deal. He's like the Arthur Holmes of game programming. [insert nerdy joke about a tectonic shift in 3D engine optimization]
Once you are done with that brief introduction you can read Valve's actual new employee handbook here. Download it, keep it. It's awesome.
I get the sense that Agile is sort of a 1:150 scale model of Valve in many ways, and I also get a sense that if/when Agile grows, you don't want to become a more traditionally structured company, so maybe this gives you an idea as to how you could grow. Or even that it's okay to keep working like this.
Let me tie this back to the original post: Openness need not stop at your choice of software. Your company can be open right to it's very core, and that can lead to success. It can lead to being the most successful company in your field.
If you have any doubt, play Portal 2 (which you can download directly to your computer via Steam). If that is a product of an open environment, and of people choosing what to work on themselves, and what happens with no management, then that is the way to operate. And you'll have fun doing science.
As an aside (and maybe this belongs in a post somewhere else), I feel that I've inadvertantly acted like a Valve employee in an otherwise very un-Valve-like company, and ironically I think doing so has brought meunique success and has advanced me to a position where I now (A) am doing what I enjoy, and (B) am filling a niche that needed to be filled. If you see a need that is not being met, don't be afraid to push a little in that direction.
Wow, I got really off topic there. Sorry for the tangent!
Great overview of opensource software. I'm going to share this with my fellow grad students. The only software I can think of to add to the list is Google Earth. It's great for making short videos to include in presentations to bring the audience to the field site.
@TSherry,
Oh yes, Google Earth is amazing as well. Those Keyhole guys were genius. I particularly like going back and forth between QGIS and Google Earth, whenever appropriate. What do you use for "short videos", any tips? I have used www.fraps.com, that comes from the gaming world, but are there other tools you find do the trick?
@Maitri: Wow, thank you — that is a great resource. How funny that he posted that on the same day.
@TSherry: Ha, how could I have overlooked Google Earth? Thanks for mentioning it. I have also been playing with the (open source) NASA World Wind Java recently, which was Windows only for a long time; I really like the demos and sometimes fantasize about learning Java and building something for geosciencey data.
@Reid: You, off topic? Come on :) Far from it actually — Evan and I talk about this 'corporate' stuff all the time, and growth (and especially about how tricky that is with a time-based consulting model). I know nothing about computer games, so it's interesting to hear about Valve; I will definitely check out the employee handbook. What you've written about them reminded me of Derek Sivers's wonderful little book Anything You Want — it sounds like CDBaby grew on similar principles.
Thanks everyone! Good discussion. Keep it coming!
No LaTeX? :)
@Matt: Ha, dear me no. The only thing we use it for is math, since it's built into MediaWiki (well, kind of built in) and is just so much easier than any equation editor I've met. For instance, I use this awsome tool all the time.
I do miss LaTeX a bit though — it's so easy to manage styles and cross-references. Word doesn't even come close. Now you've got me all nostalgic...
<goes off to install MacTeX>
haha! That online tool looks quite nice! Have you used LaTeXiT much (comes with standard installation of MacTeX)? So good for equations if you don't want to actually compile a whole document with LaTeX. It is a small GUI that you can type equations into, and you can drag and drop from it straight to powerpoint etc. The goods!
@Matt I haven't used LaTeXiT. I prefer TeXworks (Windows), just the code, no silly GUIs.
@MattHall, not really Geoscience specific but it might be good to throw Notepad++ on the list. It's great for coding, allows multiple tabs and code color schemes.
Great post, it is very interesting to see what free software geoscience professionals are using. Thanks for the heads-up on Geocraft. I was wondering whether you could give an example of how you use Gephi?
@Steve: Quite by chance, today's blog post has a graphic from Gephi. I used it to analyse the relationships between the topic tags in our new book.
I also did some of the analysis for my 'collaboration' paper in Gephi, though I used NetDraw for the images.
You can use Gephi (or another network graphing tool) to analyse any dataset in which elements are arranged in discrete classes, like people or facies, say, and in which those elements have measurable relationships to each other (eg Alice emails Bob, or Sandstone overlies Shale). If you can count those relationships, you can use graph theory to characterize the relationships. Maybe.
My top 5 opensource/budget/free/ubiquitous tools
1.Excel (what cant you do in excel?),
2.Powerpoint (crop, rotate, fill polygon from clipboard, set transparent colour - fast and easy to manipulate images to reveal the 'truth', and the majority of your work ends up presented in this format anyway.
3.&4 Google - special mention to Google Earth and Google Books . Google Earth -I frequently plot wells and other interesting tidbits, it remains very fast and there are no complicated rules for posting, try it you may find a number of wells are mis-located w.r.t the pad image. Google books stared out as me just being a cheapskate but evolved to me buying the right reference material.....almost forgot Google image - brilliant!
5.Skype - screen sharing presentations oversees has really hurt my frequent flyer points
6. Excel - I included it again.
@Dan: You're right, you can do a lot in Excel. Indeed, I've met reservoir engineers that seem to do everything in there, from solving equations and storing data to sending email and building million-cell flow simulators. Along with PowerPoint, they're like strange attractors — we tolerate horrible deficiencies (no proper understanding of units, no layers, clunky interfaces, BASIC programming) and bludgeon our way through our workflows, repeating operations again and again, swearing under our breath. Maybe it has to be that way... Here's what I wonder: does their very ubiquity mean you can't do anything truly remarkable with them? (OK, I'm biased).