The Deepwater Horizon Spill Showcases the Integrative Power of GIS

The ongoing natural disaster that is the BP Deepwater Horizon oil spill provides both a showcase for the power of geospatial technology for rapid response to natural disasters, and the conflict between data control and providing people with a timely and clear understanding of a dynamic disaster response. Drew Stephens, founder of the GIS Institute, was an instrumental actor in setting up an enterprise GIS in BP’s Houma incident command post (ICP) before being let go after 21 days without explanation. V1 Editor Matt Ball had two conversations with Stephens, about the phenomenal GIS work that is ongoing, with some details about the conflict with IT, and again to detail the steps that Stephens is involved in to make the oil spill data available to the public.  

Conversation 1: June 8, 2010

V1: When we last spoke, it was about your efforts to map and educate about the plastic problem in the Pacific Ocean (see Mapping the Plastic Problem). It’s both ironic and distressing that you have been involved in perhaps the largest ocean environmental disaster, dealing with the raw ingredients of plastic. Was it your work with plastic mapping that led to your assignment with BP?

Stephens: Yes, I was in Belize working on our coral reef baseline project, which also has a plastic component to it, when the explosions happened. We had just finished celebrating Earth Day actually. By the time I got home, I heard that the rig sank, and I had two thoughts. The first was that those are ships, they have captains and they can sink. The second longer thought was of an out of control garden hose with oil coming out of it. I couldn’t imagine where the line from the bottom of the Gulf to the rig snapped, but certainly it snapped if the rig sank.

I was glued to the TV for a couple of days, and I was very upset at my core with the tragedy of the commons that we’re entering. I had to stop watching, and stayed away from it. Then I got a call from Devon Humphrey from Waypoint Mapping, a past colleague at ESRI, telling me that I needed to get down there.

Devon got the call early. He teaches spill response at the National Spill Control School, which is mandated in the Oil Pollution Act (OPA) of 1990. The spill school is at Texas A&M University, Corpus Christi, where he teaches GIS and spill response as part of the curriculum. For him, this was the big one, the career event.

When I got there, he was so tied up in map requests, and trying to create lab space to work in, that my first job was to stay out of the way, watch, and understand what needed to happen next. Three days later, I was GIS unit leader and he had been promoted to Geographic Intelligence Officer (GIO). We instantly got as much rope as we needed to build a GIS that could support all of the mapping needs quickly, and we knew the system would also need to handle many types of requests, and scale to support massive use and size. People were walking in with wide eyes and exclaiming, “I need a map,” whether it was the Coast Guard, the National Guard, a contractor or an environmental agency.

V1: What was the state of the mapping infrastructure in those early days?

Stephens: We had flash drives, laptops, and different hard drives, with people going home at night with their own personal computers. When new data came in, we had trouble tracking the ArcMap GIS Project Files (.MXD), because they would be on someone’s personal laptop. The number one job was to centralize MXD’s and build a geodatabase. It was a mess.

I got the green light to start hiring people, and I started calling friends and colleagues. Two in particular came, Darron Pustam and Deborah Mathews.Those two sat on the far side of the GIS lab, and people would try to task them with mapping, but I literally defended them because they were building the database, and we desperately needed to get multiple processes organized.

So, here’s what we did. Every MXD that was more than three days old was backed up, and the newer files were placed on NAS drives we picked up at Best Buy. All the Shapefiles from the flash drives and laptops were converted into a file geodatabase, and we started re-mapping all the MXDs that were less than three days old. If we needed the older MXDs, we knew where to find them.

That was the beginning. It took four days to get all the data and files centralized, and another two days before we were had an SDE geodatabase running on an Enterprise Advanced ArcGIS 10 Server.  Including all the map requests and new data requests we had to meet, we went from nothing, to enterprise-class in just a few days, which was amazing.

V1: How did you go about putting the workforce together, and how were they organized?

Stephens: When I was told to hire ten mappers, I realized that I couldn’t recruit, welcome, introduce and manage that many people all at once. I had to do it slow at first, and understand the needs and then meet them. The National Incident Management System (NIMS) protocol says, once you’re managing eight or so people, you need to split that off into teams. Ideally, five people are the number that you want reporting to you directly.

As the team grew, the needs grew. I found myself with about 12 people under my leadership, and we knew we had to get the Org Charts going. We broke into teams for cartographic production, database management, geospatial data coordination, situation maps, and a growing field responsibility. The field response consisted of sending people out in the field to create maps and also to train people in using ArcGIS Server Mobile.

V1: What kind of map requests were you receiving?

Stephens: Initially there were two maps that we made daily. There was new SLAR (side-looking airborne radar) coming in two times daily from Dash 8 aircraft with imagery of the plume. We’d have new imagery of the plume that we’d georeference, vectorize and map twice a day – these were the first “regular” maps in the production cycle. People in the unified command center were getting used to those products, and there were easily a hundred custom maps being gererated by the team on some of those early days.

We made a 24-hour clock timeline to see what maps Unified Command were used to, and what map requests we were receiving. We were working to see if we could create just a couple more map products that would meet most of the demands of people coming in the door. We could then show these maps to those coming in the door desperate for a map, and could let them know when they could expect these on a daily basis. We tried to standardize the map products along the timeline of our data receivables.

In mid May, The Louisiana National Guard conveyed a need for a daily large scale map with actual boom locations, so we created a 7.5 minute mapbook series which was plotted to PDF every day. Here’s when it got really interesting… The National Geospatial-Intelligence Agency provided support of a team of NGA analysts who reviewed classified imagery manually to heads-up digitize actual boom locations that they could clearly see in their imagery, and they handed us the layer. We got two amazing deliverables from the NGA in those early weeks, the satellite-derived plume polygon of the extent of oil, and line work of booms. We could pretty much track whether those were anchored properly, blown out, or washed up in the marsh somewhere.

We already had a planned boom layer that people were depending on, and we’d get reports when it was deployed. When we started to put out this map with actual satellite-verified boom, there was often a discrepancy. People were seeing long stretches of coastline without boom, even though it had been reported as deployed. We had a verified way of saying that it was not there any more. We could actually watch boom go from deployed properly, to out of place, to washed up on a beach over a period of three days or so.

Now we had data products coming in on a 24-hour cycle that people were really interested in seeing. Every time a new process would come up, it would illustrate the need for some other process. We started to print out our map and data deliverable chart on the plotter for the wall, so we could draw on it with pens and pencils every day, adding details on what data was coming in, and when and what map products were going out.

V1: What was a typical day like?

Stephens: It’s hard to describe what happens on these incidents, but no day is like the previous. I’m aware of emergency management, my father studied it, and I’ve been around enough wild fires and hurricanes. You see them coming, they’re getting close, and it hits and then the next day you’re in recovery and have to respond after the event is over. This oil spill is like nothing I’ve ever seen, because we’re past 60 days into it now, and it keeps evolving.

Even after day 15 when I was there, the day 16 was nothing like the day before. People wanted new products, and the intensity from command, the public, and the media was growing. There were people with two weeks straight of 18-hour days that were running around with circles under their eyes, and I was telling people to go home and get rest.

Process was king. There were plenty of people pointing out problems, and my policy was that if a person stated the problem three times in a conversation, I would get really short. I’d say, “I understand the problem, I want to swim in the world of solutions with you. What can we do?”

People weren’t used to people like me saying things like that, and it didn’t matter who they were – I would often interrupt conversations with two words “ecological disaster!”. I didn’t have all the answers, but my job was to get data to the people on the ground that needed it. I said something like that to someone in front of the incident commander, and he got up out of his chair and shook my hand.

V1: It sounds like much of the challenge was around communication with different people.

Stephens: I’m a fan of the people part of GIS. In an Intro to GIS class, I used to gloss over the “people” part when I taught about the components of GIS as, hardware, software, data, people, and some kind of need or process. Of course there are people, and they make decisions. I am convinced now that the people part of this response, and I’d dare say any GIS, is the critical piece. My job was to be certain that people were hearing the issues around any particular task, and that the timing and the way people work needed to be considered in any solution.

I had people coming into the room talking so fast that I couldn’t understand what they were saying. I’d have to interrupt and say, “Hi, I’m Drew.” They’d acknowledge that and then speed along with their problem. I’d wait until the next pause and say, “I’ll listen to you if you will tell me your name.” It would fluster them, and I’d just say that we try to slow it down and take things one step at a time. A few minutes later, after being able to understand their problem, they’d walk out with a map in their hands and a smile of their face.

With other people, I’d have to ask them to tell me the problem using completely different words. Sometimes they couldn’t do it, and I’d hand them a marker for the white board and say, “can you take this and draw a picture of what you’re trying to tell me, because I do not understand.”

Many times, the person was going so fast, and thinking that the process would work, and that everyone understood it, and then by the time they drew it out they realized that the process wouldn’t work. I was helping people understand context as well as content. It was amazing and maddening all at the same time.

V1: Were you impressed by the amount of resources being thrown at the problem? Did you want for hardware, software, data or any other resources?

Stephens: If you think about it, a need is actually a resource in a funny kind of way. If these people need this, we can connect them to that, and send them out with a GPS and they can post new data back to the GIS. A lot of Devon and my jobs were to scope the situation, and develop the playbook with all the resources and the needs included.

Officials would come in just to ask if something was possible. What I was hearing on the fly from these people were new layer requirements to the GIS, and they were telling me what the schema was. We’d go to the white board with all the details that they wanted to map, and fifteen minutes later we’d have a mobile application on a GPS. We would go out the door to collect test data in the parking lot, and come inside and show them the data collected already in the server.

In this regard, ESRI was amazing with their contribution. They sent us two Windows Servers, and we put ArcGIS Server 9 on one and ArcGIS Server 10 on the other. They also sent us two of their top ArcGIS Mobile guys.

The resources were definitely there. My first day there, before I was even badged-in, I saw a cart of Garmin GPS units, and a pile of digital cameras headed out to the field.  Later, we were sending out loads of Trimble GPS receivers, and in places where there were no cell phone signals, we’d send out a mobile WiFi hotspot that linked to satellites. After Louisiana National Guard members collected data on the miles and miles of protective structures they created, they’d walk up to this battery powered device and send us back all their data. All that was happening in less than three weeks.

I wouldn’t say that people were out spending money for no reason. We tried once to order a number of servers to all be part of a replicable system between area command, the three incident command posts, and BP’s corporate office. BP and their contractor TRG already had ArcGIS Servers up and running, and we were trying to order four more servers total to collect, edit and replicate data to area command and corporate servers. That got shot down quick because it was a couple hundred thousand dollars worth of equipment. We were fine with our loaner servers.

The Coast Guard made a request, and asked us what we needed to fulfill it. At that point most of the GIS staff were working on personal laptops, and that wasn’t going well. We ordered 12 workstations and a server with ArcGIS Server on it. BP paid for that, realizing they would be there a while.

V1: As this whole incident has dragged on, with the well still leaking, and different scenarios failing, what was the mood and pace of the response?

Stephens: Have you ever seen a picture of water under the eye of a hurricane? Storm surge is a bulge of water, because the air weighs less at the center and the water swells to equalize. Then the higher water moves ashore and becomes the storm surge. If you think about that bulge, and equate that to a blob of oil, the whole incident was starting to bulge with needs and requests increasing, and the number and type of agencies needing data products increasing. Every day you think you have a handle on things, and then something new happens.

We built a Flex viewer that went viral in the building as soon as we launched it around day 20 of the incident. We just shared the IP address, and you’d walk around the building, and soon everyone was using this GIS viewer. Suddenly people are complaining about the color or the lack of a certain data set, but they’re all looking at the latest data, and the fact that they are commenting on it meant they were using it!

The first thing that became noticeable to me as a challenge was that we were building this amazing GIS with the latest and greatest technology, but people in the EOC’s in the Parishes, and in the staging areas, could not see this data. They couldn’t see the viewer, because we were on a network that was not accessible outside the building. About the same time the permanent server showed up, and BP’s IT department wanted to put this behind the firewall, and they did.

The needs were growing, and the capacity of the GIS was growing, and the GIS staff were growing. The teams were getting organized. New clients were coming into our room, with requests to use GPS units to capture their data and participate in the enterprise GIS. There were multiple groups of people creating layers in this GIS, and only from the field. It was really amazing. But I started thinking about getting these data sets to responders on the coastline, and in EOCs. The parish liaison and parish staff are calling me, asking me for help. That’s when I started speaking on behalf of the locals in these various impromptu meetings, asking how we could securely get data out to the staging areas.

V1: What kind of response were you getting to that request?

Stephens: IT was looking at it. It’s almost as if they didn’t want to touch it, because there’s no way to deliver data outside the building and keep it behind the firewall at the same time.

I started to realize the security challenge as we’re studying this National Incident Management System structure that basically says, responsible parties, federal and state coordinators, will share copies of data to support the timely flow of information. BP’s IT department didn’t appear to understand the requirements of NIMS, and I felt they were trying to build a business unit.

There were people in the building who apparently were not fully aware of the protocol for an incident of national significance. There were people in Houston making decisions about how this GIS would work that had no idea what it was like to have all these requests for data and information products in certain timelines. IT staff in Houston asked us to freeze the schemas, which was very challenging, as some of the user data requirements required new schemas or new fields at a minimum.

By the end of my second week, my understanding of our job was that we were to help IT (still new in the ICP) understand what we were doing, so that they could follow and support us, but a stoppage in responding to an ecological disaster for a freeze of schema so they can replicate with Houston was not a priority for me as the ICP GIS Unit Leader. There were compromises and setbacks, and yet we always seemed to keep moving forward. There was simply an architecture conflict combined with a misunderstanding of vision and technology. It’s my understanding that all the people involved with the dialogue didn’t understand what was happening, including me. My sense is the conflict of vision and misunderstanding led BP to ask us, through our contractor, not to return to the building.

V1: Do you know how the operation was affected by this?

Stephens: The next day, they were without six of the core GIS team; Devon and I removed, two turned-in their resignations, and two more were rotating out anyway. It happened May 28th, and I’ve talked to someone that was there who saw what happened. It sounded chaotic at first, with new contractors running the lab, yet I suppose by now they’ve figured out and improved on what we built.  There was a lot of work happening that nobody had time to document.

I wrote a letter the next week about my concerns, and I sent it to the White House, I sent it to Interior, I sent it to the incident commander, and I sent it to the Coast Guard, and other concerned individuals. I posted it on my Web site, and there was enough misunderstanding about it from conspiracy theorists to everyday people that were upset, that I chose to take it off the GIS Institute site, because our core philosophy is to identify the problems and start solving them.

I felt that the reaction to the letter was improperly labeling me and The GIS Institute, because people seemed to want me to stay in a battle mode. That letter went out to people that needed to see it, and I know some things changed. I’m moving on to fix what’s still wrong from the spill, and I’m thrilled to know I can still contribute to the solution.

I’ve heard that the Coast Guard is replicating the BP Server. I don’t like that the US Fish and Wildlife personnel are still posting data to a server behind the BP corporate firewall. It just doesn’t look good, and in my opinion that’s not the right way to do this response My opinion is that a neutral third party should be collecting that data off those mobile devices, and replicating with the other servers in Unified Command.

V1: This incident is so unprecedented in terms of its ongoing growth and impact. The mapping of the space shuttle crash was impressive for the multi-state swath of the debris, and the effort to collect and map it, but this is ongoing, and it’s going to be with us for years and years. The cleanup is going to take a long time.

Stephens: This will be around for a long time, and we don’t know how horrific it will get.

Now we’re talking 60,000 to 100,000 barrels a day, and we’re now two months at this rate. You kind of step back and think it’s too late -- too late to set up the best technology, with the smartest architecture ever, to record and track this disaster with a geospatial baseline. But it isn’t even close to too late. There’s still time, and its okay if everyone has their own viewer. But there’s still time to make this data accessible to everyone that need them.

Every day we ran the map book routine that cranked out a 1:24,000, quad-sheet indexed, verified boom location map into PDFs (they were out of date ‘before the ink dried’), and all we could do was stick those up on an FTP site consumers to download. If we had viewers with full data access, responders could just access the viewer, zoom to their area of interest, and print custom maps or download the actual data for their study. Instead it’s this crazy old approach (FTP – Forgot To Post), and it’s a bummer that we’re relying on that instead of having a viewer or map service, reading a database directly, available for consumers. Even the ERMA viewer needs to be ‘fed’ from other databases.

I’m convinced that our progress was hindered by people that just didn’t understand the technology. We were slowed down by questions and tasking that didn’t necessarily need to take place. What I’ve learned is that the architecture that we originally proposed, is slowly being adopted for all the right reasons. I’m a little disappointed that we didn’t get to continue working on this dream job, but I’m sure it’s all happened for the right reason.

Conversation 2: July 5, 2010

V1: What’s the latest? You mentioned last time that we spoke that you had a few different options in the work for setting up a neutral public-facing server. Has that solidified?

Stephens: We’re moving forward with a software donation to the GIS Institute from Jack Dangermond for ArcGIS Server and a hardware donation through a vendor, and we’re partnering with Appalachian State University to host the server. There are two professors there (John Pine and Chris Badurck) who are studying how GIS data are being generated, consumed and how the data flows in a rapid response. They are going to be applying for longer term National Science Foundation grants to understand the long-term impacts of the spill.

I’m also working to use the server that we set up for a public-facing GIS. We’ll have a direct viewer set up that allows people to download small areas (for instance a county) and download that data on a layer-by-layer basis in whatever format they choose. We’ll also include tools for some basic analysis. For people that want to know more than pan, zoom, identify and measure, we’ll have some pre-loaded analysis tools to allow people to select views and features to do analysis through the viewer.

Then we’ll also have a supporting page for the viewer that will include input of social media and web-cam type content. What we’re seeing is a lot of Twitter, YouTube and cameras being used with coordinate information attached. We want to make sure that all the social and public interest media is supported using the same operational data that Unified Command is using. We’ll see their boom, their oil plume, and we’ll be able to overlay and do mashups such as weather and current information from a variety of sources.

From what I understand, we’ll be able to replicate with the data from Unified Command, with a viewer and database that is geared for the public and researchers, and for parishes and emergency operations that aren’t necessarily part of Unified Command.

V1: Has the data arrangement been fairly straightforward at this point? Was there much back and forth to put that into place?

Stephens: If you recall from our interview two weeks ago, there was a lot of misunderstanding between some of the GIS players, particularly myself and BP’s IT department. What I know about being in the response, particularly in the building where people are responding to this disaster, is that what’s right in front of you gets your attention. What I know being away from it for a while is that everyone is doing the best they can, and we’re all missing bits and pieces while in combat mode.

Perhaps the letter that I wrote helped shake things up a bit, as it went to some interesting places. What I know now is that BP is allowing other entities to replicate directly. The U.S. Coast Guard has a server that is replicating with BP, so as the federal on-scene coordinator they’re seeing the same data that BP has.

Our hope is to do two more things. One is to set up this server to inform the public and allow access for researchers and other responders. Louisiana State University has a server that we’re also going to stand up so that there is a state component of the triad of federal, state, and responsible party from NIMS. The three parties will all be replicating, and then we’ll stand up this extra piece, for the public and everyone else.

We’re still working out exactly how these things are going to happen, but we understand now that we are going to be able to replicate to serve the public. There’s an additional node that may play into this. The Texas A&M, Corpus Christi University where the National Spill School is located, will play a role to replicate the data and mirroring the public access work that we do.

I wanted to open up the data immediately to the public and others when working on the response, but I understand a little better now why that couldn’t happen right away. It’s a huge undertaking. Now that I’m away from the command post, I feel quite passionate and enabled to make the public GIS available.

V1: Are there specific applications or focus areas for the different efforts?

Stephens: There are specific research areas. We’ll be looking at where the oil is or was, and the long-term impact of that over time. We want to literally see how the marshlands and fisheries were affected, even into the economic and social impacts. The Appalachian University team will be looking at the long haul, with remote sensing and economic and social data. The viewer will have a number of different applications, and we’re not quite ready to start listing those yet.

V1: Are there any benefits that have come from your involvement in the spill response?

Stephens: We started the first interview talking about my work mapping marine plastic pollution. If there’s any upside to this, and I don’t think there’s much, it’s that we’ve been able to get attention and interest for the far end of this hydro-carbon cycle. Along with the ESRI software, and a NSF hardware donation, we’re planning on deploying a global plastic pollution mapping application. This will be hosted by the Institute at first, and my hope is that it will be so successful that we can’t handle the bandwidth.

It’s a plastic application that you can download onto any Windows Mobile device, and soon we hope the iPhone and the iPad. The idea for this came from the experience of deploying mobile GIS for the Deepwater Horizon response.

There will be two pieces to it. The first is a way for conservation professionals to login and contribute scientific-grade data to this map. The second tier is a true community mapping effort and that’s one for anyone anywhere to contribute to a citizen’s science-based map. People that go out and clean a river, a lake or the ocean, can all use the same app to contribute to this global plastic map in a spirit similar to Wikipedia. We’re going to trust that there are good data.

I feel that in the geo community people are much more willing to bring in social media and community data. I think there has been a reluctance to allow anything but verified, and high-end data to these systems, and that’s why we’re going to do both. We’re going to enable both volunteers and scientists.

V1: I understand that you’ll have a strong presence at the ESRI User Conference next week. What can people expect to see?

Stephens: We will be focused around this plastic mobile GIS capability, our outreach for the public to the Deepwater Horizon data, and our work in Belize where we’re doing a lot of baseline mapping of the marine and terrestrial environment. We’re doing work on an atoll that’s 50 miles from Belize City.

All three of these major projects really embrace the core and brand new technology of ArcGIS. Allowing people to understand how these technologies work is the most important thing in having them used effectively. The people part of GIS is the primary component.