INTGeoServer

Jul 16 2020

Jumpstart the Development of Your Next Cloud Application with GeoToolkit.JS and INTGeoServer

The Oil and Gas industry is turning to the cloud for its digital transformation. In the race to revolutionize E&P, companies are faced with a chicken-and-egg problem:

How to build cloud-based applications when the data is still within the confines of the company network?
Why move to the cloud when there are no applications that are able to use this data?

INT has been a long-time pioneer by providing JavaScript components that empower developers to build geoscience applications that run in a browser. The GeoToolkit.JS libraries cut years of development time for any company creating a new application or replacing a legacy system. However, the added value of this kind of application is not just in accessing and visualizing geoscience data, it’s also in the integration of the company’s knowledge within this application.

While GeoToolkit.JS provides the tools to visualize geoscience data, INTGeoServer provides the tools to access remote data. This server has been designed to serve seismic and well data efficiently to web clients. It uses the HTTP protocol and works natively with your existing files (such as SEG-Y, SEP, and LAS). In just a few clicks, you can have a running instance of INTGeoServer, upload files to the cloud, and visualize them immediately with GeoToolkit.JS.

Most customers using INTGeoServer elect to install several instances. To work efficiently with seismic data, INTGeoServer needs to be close to that data. Since E&P companies have their data scattered all over the globe, so are the installations of INTGeoServer, allowing access to datasets from multiple sources in one application. In a classic configuration, data ubiquity is typically achieved by deploying worldwide file systems. INTGeoServer optimizes remote data access by applying several techniques that networks cannot use: only sending the data that the GeoToolkit.JS client needs through the network, limiting the round trips, and compressing the data leveraging similarities between adjacent traces.

GeoToolkit.JS has a built-in API to access INTGeoServer instances. It only takes a few lines of code to program a JavaScript application that will read remote data and visualize it. As a result, programmers are free to focus on the added value of their application.

INTGeoServer also offers a nice transition from classic file systems to cloud-based storage. From the perspective of the web client, the code is storage-agnostic. While a company works on migrating its data to the cloud, its developers can use instances of INTGeoServer that are bound to the company network. Once the cloud is ready, no changes to the application are required. You do not need to decide in advance which cloud provider will host your data. INTGeoServer works with Amazon S3, Microsoft Azure Blob Storage, and Google Cloud Storage. If your application serves data from multiple vendors, you can let each vendor choose their own cloud.

GeoToolkit.JS is meant to empower developers. It provides ready-to-use components that can be customized by developers outside of INT. Similarly, INTGeoServer is a platform. It has an API allowing you to add your own data formats, your own security, and your own science. As the audience of your application grows, you might elect to implement your own data server. INTGeoServer facilitates this future transition by documenting the HTTP protocol it implements. You are free to implement your own version of this protocol, keeping your JavaScript web application running without requiring any changes. In this particular scenario, INTGeoServer gives you a definitive time-to-market advantage.

GeoToolkit.JS allows requesting seismic data, log curves, trajectories and horizons form INTGeoServer. The following screenshot displays a cross-section display built with data located on INTGeoServer.

LogCurve can be requested using a simple REST API from INTGeoServer and visualized inside WellLogWidget or MultiWellWidget.

Seismic data in different formats like SEG-Y, SEG-D, SU, and others can be indexed by a utility provided with INTGeoServer, and GeoToolkit.JS can leverage it using sophisticated queries. It is easy to request seismic sections using RemoteSeismicDataSource and specify an arbitrary path or INLINE and XLINE to get data located in cloud or private storage. Moreover, the seismic volume can be visualized in 3D with the Carnac3D module of GeoToolkit.JS.

As the industry continues to shift towards a digital transformation, more and more E&P companies will migrate their data to the cloud. And with the support of GeoToolkit.JS and INTGeoServer, it becomes simple and efficient to integrate, access, and visualize a company’s data within an application in the cloud.

For more information about GeoToolkit and INTGeoServer, visit the GeoToolkit product page or contact us for a free trial.

Aug 09 2019

In Retrospect: 10 Years at INT

This month of July marks a significant personal milestone since I have worked at INT for 10 years. 10 years is a long time, especially in technology where paradigm changes occur approximately every three years. Yes, the word “paradigm” was actually in vogue the year I started at INT—that’s how long it’s been. For this anniversary, I’d like to take you on a chronological tour of my experience.

The Formative Years

The first two years at INT were spent learning the many aspects of the application and the science I was working on: INTViewer and subsurface data. I liked joining a new team and getting acclimated to a new code base. I learned a lot from INTViewer’s architect. For example, he helped me understand the significance of making aspects pluggable. Not only does it serve INTViewer as a platform, but it allows the code to evolve without getting out of control. Following this principle, INTViewer’s code base has been able to grow several folds. And we’ll see that the plugins approach served me well in other projects over the course of 10 years.

Growing with INTViewer and INTGeoServer

After the first two years on the job, I picked up more responsibilities. Becoming the “ultimate resort for INTViewer questions” affected me in a way I didn’t anticipate. When I first started, whenever someone asked me a question I could not answer, my internal dialogue went something like, “I don’t know that part of the system. Who is the best person to ask for help on this?” After two years, this changed to: “I have been in this situation before. I know I will find the answer.” This somewhat irrational belief that I can answer any question thrown my way has helped me quite a bit when it comes to solving problems and helping others. When a coworker has a tough technical question, I didn’t anticipate I would one day answer, “Let’s find out!” with such confidence.

The needs of INT’s customers have changed over 10 years. One particular concern that has been pervasive across that time period is the ubiquity of data. Before “cloud” became the new word in vogue, customers often came to me with this problem: “I have teams all across the world, but I don’t want to maintain a worldwide file system. Visualization needs to be fast for all, without having to duplicate data. What do I do?” It’s out of these conversations that INTGeoServer, another pluggable platform, came to be.

INTViewer had years of experience built in to how to access data files efficiently, but, as a product, it needed to move beyond the file system. This was a complex technical challenge and an opportunity to widen the team’s technical skills.

INT gave me other opportunities to innovate: the integration of Python with INTViewer is quite unique in the market. Looking back, even though the technical solutions to reach “data ubiquity” have changed over the years, even though we introduced new ways to automate geoscience workflows, the fundamental work on geoscience data hasn’t evolved much. While software can be a scary place with its rate of change, I find that the geoscience learnings from my first two years are still relevant.

Building IVAAP and the Future of Ubiquitous Data

The latest evolution of ubiquitous data is cloud-based. The last three years have been a sort of new beginning for me since I’ve been tasked with leading the data side of IVAAP. Most of the IVAAP backend was essentially written from scratch, which is very satisfying as a developer. What is even more satisfying was working with the development team and seeing it grow. Since the backend was written by this team, there is no longer an “ultimate resort for questions” role. With the recent work with the OSDU consortium, I am happy and proud that the architectural decisions we made over the last three years have shown we are going in the right direction. This was recently validated by making IVAAP compatible with the OSDU platform. This work didn’t require any changes to IVAAP’s SDK—IVAAP’s OSDU implementation is actually just a plugin for this backend.

A Developer Culture

Working on INTViewer, INTGeoServer, and IVAAP for a grand total of 10 years, what has made me show up every morning has been the deep technical aspects of the job, the products I have been able to work on, and the people I interact with. INT has been a wonderful opportunity for me because of its technological leadership. If a developer says “I need X to achieve Y,” this gets immediate attention because the company culture is very developer-friendly. If you are a developer at heart like I am, being able to write code all day without interruptions is a significant perk of the job. Frankly speaking, these 10 years were also possible because developers at INT are seen as an investment, not a cost. Unlike other companies, INT has a strong will to weather tough economic cycles without shrinking its staff. I have grown with INT, and we both keep growing together (we are hiring, by the way). As a leader, I strive to help today’s new hires to have the same positive experience I had.

Dec 01 2017

How to Improve Performance and Reduce Latency of Your Geoscience Data

Storing and accessing large, sometimes sensitive geoscience data is one challenge many top E&P companies face.

Local access is great, but not every user in the world can have local access to the same data. Replication is an option, but with the size of seismic datasets reaching terabytes, this is not practical. In the real world, users only have access to a limited set of local data.

Common Solution Leads to Performance Issues

Many companies store data all over the world. The common infrastructure to allow ubiquitous access to data is to share these files using NFS, a well-known distributed file system protocol used by Linux-based servers.

The issue with NFS is that it is a “chatty” protocol: Many messages are sent back and forth between the client and the server. This is fine when all machines are physically close to each other, but the further away they get, the more latency you introduce. As a result, performance degrades.

NFS is also essentially transparent to the software using it. Some software, like INTViewer, doesn’t “know” that your data is remote, so it can’t optimize its data fetching strategy to the characteristics of your infrastructure. Actually, for seismic data, it assumes that access to individual traces is fast.

A Better Option

This is where INTGeoServer comes into play. Access to data hosted on INTGeoServer—a server with a modern architecture that uses web services to stream geoscience data—is optimized so that there is a limited number of back-and-forth messages. In other words, by installing INTGeoServer next to your data, you make this data accessible from remote places as if it were local.

To visualize any geoscience file in INTViewer, simply drag and drop that file from the file system to INTViewer’s main window and its content appears automatically. From this experience, it might seem that INTViewer is tied to the file system where it resides, meaning it can only read data from that file system. While this is a common use case, using INTGeoServer removes the requirement to have INTViewer and your data on the same file system.

From an INTViewer user point-of-view, the protocol used to access the data doesn’t change the interaction—the visualizations are the same, the analysis tools work the same way. From a system administrator point of view, however, the burden of maintaining worldwide NFS shares is lifted. And the benefit of accessing that data in larger chunks is that the performance profile improves substantially.

For more information about INTViewer and INTGeoServer, visit the INTViewer product page, or contact us for a free trial.