We recently introduced JavaScript integrations to Decipad. It's a flexible way to connect data sources to your notebook and create interactive plans, models and reports that update automatically.
You have the freedom to write your own logic and return any JavaScript object. We take care of the rest. The output is converted into a Decipad result allowing you to weave connected data into your data story, from a market analysis to a monthly report:
.png)
But, enabling users to run custom JavaScript code presented a few challenges. We wanted to ensure the highest security, preserve user privacy, and provide quick and snappy responses.
We evaluated different approaches, but we wanted something that was safe, fast, and flexible for our needs.
We explored serverless functions as they offer robust security measures. However, relying on external services to run user code raises potential privacy concerns. And, this would mean integrations would only work while the user is online.
Another challenge is speed. Although serverless functions are generally quite fast, they have a slow cold start time. This means we would not be able to provide our users with quick, snappy responses.
For these reasons and a few others, we held off on this approach (for now).
Running user code within the browser offers the advantage of better responsiveness. The browser now even allows for code to be run on threads, meaning the application does not have to freeze while we run the user's code.
However, there are a few problems with this approach, most importantly, security. How do we run user supplied code, in a way that doesn't allow bad actors to write code that could severely affect the user?
To solve this problem we needed some sort of sandbox, a way to only allow certain Javascript primitives and disallow access to local storage, the DOM and other potentially vulnerable global objects.
We explored solutions that provide this functionality, but none quite met our needs for flexibility. A few of our favorites:
Inspired by these approaches, we decided to create something new that met our needs of safety, speed and flexibility.
SafeJS is a small javascript library, that runs user provided code in a web worker.
SafeJS uses a whitelist system, where we declare what we want the user to have access to. It also overrides methods such as console and fetch.
The solution is simple but elegant. All we are doing is restricting the users global context, executing the code in a worker (which already is a sandboxed environment by itself), and returning the result of the evaluated code.
SafeJS contains no dependencies, and relies on nothing but default browser APIs to work.
SafeJS allows us to run user code in their browser, providing the snappiest experience possible, while controlling the execution environment.
SafeJS also provides a controller class, which provides a nice interface for developers to work work with, as well as some nice to have features such as:
Once data is retrieved, you can work with it like any other data source or variable in Decipad. It's a safe approach that provides the flexibility we needed.
The SafeJS Javascript library is only a few kilobytes in size. It allows us to run user code in their browser and provide the snappiest experience possible, while controlling the execution environment.
We made SafeJS open source so other people might use it, and so our users can see what we do with their code.
We would love any feedback or thoughts. You can learn more about integrating data into your notebook using JavaScript in our documentation. And, here's a simple notebook exploring the price of bitcoin to demonstrate.
We’re building Decipad to make data modeling more accessible to more people. While javascript is flexible for retrieving and manipulating data, it's not a language that everyone knows or understands how to use.
We've released our first iteration of our AI Helper for Javascript integrations, making it easier for anyone to use. We’ll be sharing more details about our approach in our next post.
If you are interested in exploring Decipad, you can sign-up and join the beta here.
Cheers,
John and the Decipad Team
Stay in the loop with updates from our team and community.
Once a month.
.png)
