Safe Software Tackles User Challenges and Adds Functionality with FME 2010 Release

V1: I see that you’ve continued to improve the speed and ease of use of the product, and I really enjoyed Don’s blog post about the commitments you make to users to improve the product with every release.

DALE: We have a large install base that looks to every release to help them with the work that they’re doing on a daily basis. We’ve done all kinds of things this time to be more scientific on how we address some of the workflow issues and make our users more productive by being able to quickly create more efficient workspaces.

For example, a common thing that people do is to test data as it moves goes through FME to route it to different places or to adjust records in different ways. That whole flow direction or filtering capability was fine tuned in the latest release to make it even better for our users. We took a careful look at what people were doing and we came up with a new test filter where you can combine a number of tests into one. These improvements affect the productivity of all our users.

DON: We have a big database at Safe that keeps track of every user request. In this release we almost hit a thousand user requests incorporated into the product, and there are many more to come.

With each release of FME we look to make the product even easier to use. Another productivity improvement that we’ve introduced is a new Quick Add feature that allows users to just start typing to addtransformers.. Users can even insert an entire chain of transformers without reaching for the mouse or dragging and dropping. The number of mouse clicks goes way down, and of course we all know that when we take our hands off the keyboard to reach for the mouse our productivity goes way down.

We’ve already seen a good response to this new feature. Last week I was helping someone learn something new using FME 2009 technology and it was painful because once you get used to this new Quick Add, it’s really impossible to go back.

DALE: In order to scientifically address user issues, we put improved statistics gathering into FME for those users that install and indicate that they will share their use data with us. One of the things that we’re collecting is how they’re placing transformers. FME 2010 has only been available in Beta for a month, and already more than 40% of the transformers being placed are being done with this new approach.

V1: How has the performance of FME improved with this release?

DON: We take pride in making FME fast, and with each release we make it faster. We really need to do that because the data sets are getting larger and larger and users are doing more complex tasks as they try to focus more on data integration. The number of data sources is exploding. We now support more than 250 formats with this release.

V1: With so many formats, are there ways that you address data integration to aid the user in format choices?

DALE: We’ve added a workspace where you don’t need to specify the format of the data, the software just figures it out. That gets back to the whole dynamic and generic approach that we think will allow us to solve a whole class of problems that we weren’t able to solve before. Before we were really focused on situations where you knew the format and data models ahead of time and you wanted to adjust those.

There is a set of problems where you might not know the structure of the tables until runtime or you might not want to know until runtime because you don’t care. We can now handle these situations with a single workflow rather than requiring a specific workflow to meet each data structure scenario; we call this being dynamic. Generic is the other approach where you may not know what format you’re reading or writing because the user is going to tell you what kind of file they want at runtime. FME 2010 can easily handle both of these approaches with no productivity hit to the user.

These are things that make the latest release valuable to the existing users of FME.

V1: I’m strongly interested in the 3D and infrastructure focus of the release. Is the interest in these capabilities growing?

DALE: We think 3D is a very important strategic direction because of the increasing need in the market to integrate 3D information with other spatial data. This has been an amazing release for us in terms of the new heights that we’ve taken 3D. For example, we now offer support for addition 3D formats including Autodesk 3ds, COLLADA, Google Sketchup and Presagis OpenFlight.

However, we have to make sure that we don’t forget about existing users as we push the front into new spaces and attract new users.

DON: If you use 3D in 2009 and now use 2010, it’s like you’re using 3D for the first time. In some cases it’s as much as 100 times faster.

DALE: The other huge thing is that we’ve decided that it does matter what things look like and we bring over the textures. In FME 2009 we were kind of in denial that textures weren’t important and one of our senior staff members overruled Don and I, and surprised us by adding textures. Of course once we saw it, we said that we must have it. It wasn’t something that we defined as a priority, but I’m really glad that it got added.

With these new capabilities, we can synthesize very compelling 3D city models out of 2D ingredients. (see the City of Gavle example on fmepedia)

DON: We have another example where you have a CAD drawing with building footprints and the building heights known, as well as the location of lamp posts and other street furniture, but the orientation of the lamp posts isn’t known. Using FME 2010 in a simple workspace, you can orient the light standards so they’re hanging over the road. With the building heights, we can place them on the terrain so that the model looks like a real neighborhood.

All the ingredients are in the form of vector, raster (both imagery and digital elevation models) and textures, and 3D SketchUp models, and we integrate all of that and put it all out to KML, GeoPDF, Geodatabase, and any one of the formats that are appropriate for a whole cityscape scene.

We think that this is going to take FME into a whole new market –the modeling and simulation market.

DALE: It also will help cities get a starting point for a 3D model. I wouldn’t say that the model that we create is out-of-the-box absolutely perfect, but it’s 85 percent of the way there. Instead of starting from scratch, they can now spend their last 15 percent cleaning up their model using tools from say Autodesk (LandExplorer) or ArcGIS 10, which is now a full 3D GIS.

V1: Are there any thoughts to productize some of this functionality into a separate tool for users that are very 3D-centric that may not need the other capabilities that FME offers?

DON: Internally, we’ve certainly had many discussions about ways to productize things. At this point, it’s just part of FME. One thing that we’re looking at is entire scenarios on how we can build scenarios as plug-ins or extra value solutions for FME.

DALE: As I’ve made a few road trips and shown people these 3D city creation scenarios, it’s been interesting because the people that see it immediately ask if the workspace is included if they buy FME. There’s a recognition that FME provides a workspace that lets people accomplish valuable things, but there’s also a lot of value to the customization of workspaces from users that really know FME. At the moment we’re not selling separate workspaces, but we’re looking at ways to maybe commercialize or package those for people to discover and use.

V1: Building detailed city models bridges both CAD and GIS. Are there other developments in the CAD to GIS workflows?

DALE: We’ve just added Civil3D, and there’s a strong appetite out there for people to take that format and bring it into GIS.

DON: We’ve also extended our CAD support to be able to support 3D.

DALE: The latest release of Bentley Map is what we call FME enabled. If you have Bentley Map and you install FME, there’s a dialogue that allows you to import or read from FME and suddenly you’re able to import CityGML or any of our other 3D formats. Or you can manipulate your data and write it back out.

V1: I also noted in your release that you’ve added support for Cloud computing.

DON: We’re working with WeoGeo and they’re hard at work with our FME Server capabilities to provide solutions in the Cloud. The other big story on FME Server is security. In FME 2009 there wasn’t security and you had to build it from scratch. FME 2010 comes with a full security model where organizations can control who has access to their data, and can control who is allowed to publish to their server as well as who is allowed to administrate their server. Now an organization with FME 2010 can use the security that comes with it or plug it into their own security infrastructure.

V1: FME Server seems to be among a growing trend of server-oriented geospatial tools, but I believe it’s also unique in the problems that it addresses.

DON: Sometimes people confuse our server with ArcGIS Server, but it’s very different. Our server is focused purely on data delivery. At the end of the day, what’s being run is a workspace, whether you’re uploading data and getting a result or getting data back.

For example, within an organization you can run an FME script such as data validation and you can now put it on a website where anyone in the organization can use it. Whereas in the past you’d have to have access to a desktop machine running FME.

There’s no other software out there that does what FME Server does. We focus on the data movement aspects to make data sharing as easy as possible.

V1: I’d imagine that this solution solves a considerable amount of time.

DON: Absolutely, because now any number of people can have direct access via a link. They don’t have to exchange data via e-mail or log onto another machine.

DALE: While Don and I think that everyone should be able to be an FME expert, lots of folks aren’t yet they have the need to execute or run the types of transformations that FME does. What happens in an organization is that you have the specialists that create the workspaces so that the average Joe can go ahead and run and use them. We’ve kind of decoupled that.

When I was in Australia several years ago a user told me that what Safe needs is an FME runtime, and I asked the user what he meant by that. He explained that someone who authors a transformation should be able to bundle that up and deploy it throughout their organization so that other colleagues who aren’t FME experts could make use of it. In a way, FME Server is exactly that. Someone who’s familiar with FME pushes a workflow up to a central spot where other people can make use of the service.

V1: I really like the idea of FME Server and its ability to feed services. With ArcGIS 10, and all its enablement of web services, it seems like it will bring a whole new wave of customization. How do some of these services feed into the ability of developers to create sophisticated tools that will only be used online?

DALE: We’ve got a lot of tools and capabilities in FME Server that makes it a great aggregator of different services, including ArcGIS Server. We can pull together a number of different data sources, add value and then spit out a result.

DON: That really goes back to some of our user requests to make FME Server a better web client for web services. We have a category of transformers in FME called Web Services that we’ve really expanded a lot. The whole idea there is that someone has a website they want to get information from, and we don’t care if the user wants it to give back HTML, XML, GeoJSON or whatever. We’ve made it very easy for our users to use those web services so they can incorporate them in their workspace.

V1: One other thing that I noticed is that there were a lot of statistical formats that are addressed in the new release. Is that a trend in order to enable greater analysis?

DALE: We came out of the gate with three of the more popular statistics formats. This statistics push was done in partnership with a company in Seattle called Circle Systems that focuses on moving statistics data. Their product is a lot like FME, but for statistics. By teaming with them, we are able to harness all of their efforts and open up FME to solving a new set of problems.

We had people from all over the world asking us to integrate SAS data, SPSS, and R. I think these are workflows where they have spatial data with a fairly rich attribute set, push it over to the stats side, do some grinding and analysis that is non-spatial and bring it back and put it into something else to show the results. We’ll see how that unfolds, and see who takes advantage.

As an aside, if you are only a stats person, FME could be considered a good ETL for statistics. There aren’t very many tools that allow you to go to even the same stats package while rearranging data along the way. The stats transfer people are really a file transfer mechanism; they don’t do the transformations that we do. It opens up another front for us, and in our next release I’m thinking we’ll add even more stats formats.

V1: Right now we’re faced with so many issues that require us to analyze change and what’s going on, so I’m excited about these kinds of forays that open up people’s imagination.

DALE: I’ll tell you one of the applications that I’m sure is close to yours and Jeff’s heart. One of these stats initiatives is related to the Swiss Meteorological Service that wanted to use R to do statistical climate change analysis on data that is otherwise stored in SDE. That’s a scenario that’s exactly what you’re talking about.

V1: A lot of multidisciplinary workflows are being addressed now, and I really like the idea that you can bring a lot of disciplines together even if they don’t work with a common toolset.

DALE: I was at the FOSS4G event in Australia, and one of the speakers was a world-renowned climate scientist from Australia who has done a lot of computer models. It kind of shocked me that all of these climate models are being done in Fortran code. If all of these modelers are having to build importers to bring different data into Fortran, it’s got to be close to the most awful programming task that you’d want to do. If we can play a role in helping them get access to their data faster, and spend their time productively analyzing and creating models, I think that’s a good thing.

V1: It seems that we’re becoming more aware in the geospatial community that there are so many other models and modelers out there.

DON: That’s really the strength of FME too, making these different data models work together so that you can move from one to the other. The data model is the key. Now you can actually do a workspace from one data model to another data model independent from the format. In the past you were really tied to both the format and the schema. Now with FME 2010 you can only be tied to what you want to be tied to.

A real example of an easy one is that now you can build a workspace to do reprojection to add a length or an area to every area feature independent of the format or data model.

V1: I’m excited about a lot of these new features, and about the work that your work enables.

DON: We’re pretty excited as well.

DALE: It’s one thing as the engineers and creators of all of this to live and breath this stuff, but what’s more exciting is to see what people are going to do with it. We’ve already had a taste of what some folks have done with this, but now that it’s rolling out we’re really excited to see more new work.

Don and I are taking to the road in March to be face to face with users. We’re doing the East and West Coast of North America simultaneously in a series of events. We’re calling it, “2010: An FME Odyssey.”