Automate Python scripts in Retool Workflows | Retool Blog | Cache (2024)

Today, we’re introducing support for Python in Retool Workflows. Now, you can run Python code blocks as steps in your automations to more elegantly ingest, analyze, and visualize data—with preinstalled dev tools like autocomplete, linting, and 20+ libraries (like numpy and pandas).

We heard from customers that building business software is a collaborative effort, and support for the languages that their teams already use helps them build apps faster.

We want Retool to be the fastest way for your team to build, maintain, and extend business software. So we’re doing just that, starting with Python support in Workflows.

A lot of software today is written in Python. (GitHub’s State of the Octoverse ranks it the second most popular programming language used in 2022, behind JavaScript.) And for good reason—Python comes with a rich library ecosystem, which you can now leverage when building automation projects in Workflows.

Now, when running code blocks, you can select either JavaScript or Python as your language. Just drag a block out onto the canvas, select “Code”, and then select the Python tab to start writing.

When writing Python in Workflows, you automatically get auto-completion, context, and linting. Don’t worry about importing your favorite libraries, either. We’ve preinstalled 20 libraries and packages—like numpy, pandas, scipy, and plotting libraries like seaborn—so that you don't lose access to the powerful packages you’re used to writing within your IDE or notebook.

As someone who does a fair bit of analytics work, there are times I find myself wanting to reach for something other than JavaScript or SQL when building in Workflows. Applying normalization to a multi-dimensional dataset feels needlessly complex in JavaScript when, in Python, I could simply call np.linalg.norm.

Now, I can swap between JavaScript and Python any time I want to use code blocks in Workflows, meaning I can reach for the right tool for the job at each step in any automation process. Here’s a reporting example:

Say we want to create a weekly report for our Customer Success team, tracking their healthiest and least healthy accounts. First, let’s start with writing the base query to our dataset: in this case, our table is called fct_sfdc_account_health.

Getting the base-level data using SQL is always a great first step, and with Retool, you can use your existing SQL resources to quickly query your database. But if your reports need to be customized to each user, generating user-specific SQL queries can get annoying fast. Instead, after doing a generic SQL query, we can handle the transformation and filtering in pandas.

We’ll use pandas to quickly sort, transform, and clean the data. Our Success team is only assigned to certain segments like Enterprise (ENT), so we filter for those here. Additionally, we can truncate any customer names that get too long.

We won’t show all of the transformations here, but you can imagine the utility of writing native SQL then native Python in back-to-back blocks. If your logic is highly conditional, the graph-based expressiveness of Workflows supercharges your developer workflow.

Finally, we can use an email resource like SMTP to easily send an email to relevant accounts and schedule the workflow to run every Monday morning. We can even leverage the trick above to add an attached PDF containing a visualization we generated earlier:

Like software development, data is a team sport. Data analysts and operations teams collaborate with engineering on everything from product metrics to reporting dashboards, and maintain operations software to run the business as a joint effort.

Workflows turn every script or data processing workflow into a portable, reusable API to easily leverage complex logic in your Retool UIs—facilitating quicker handoffs between teams without having to worry about provisioning any additional infrastructure.

For example, let’s create a webhook-based workflow. By flipping the Start Trigger block to webhook mode, we’ll automatically get a URL endpoint that we can send POST requests to. And, we can specify the shape of a JSON input to reference later—for example, the attributes on various axes, or filters to apply.

Just like the reporting use case, imagine that we’ve read in data from a SQL database. We can then use pandas to filter the dataset based on the parameters provided in the startTrigger.

Now that we have our desired dataset, we can use a Python-based plotting library like seaborn to create a nice plot.

We’ll create a file buffer in the Python block, then save our visualization into that file buffer. We can now return that image (in svg/pdf/png/jpg format) as the output of the block. We’re almost done!

We can use the new webhook Response block to simply return the base64 version of the image generated above.

Now, we’re pretty much done. Jumping over to the Retool app editor, we can create a form with all of the relevant inputs needed for the chart.

On submit, we can call the endpoint that is exposed in our Start Trigger.

And finally, we can add in an Image component with the URL directly pointed at our REST Query above.

Python code blocks are now available to all cloud customers and will be available on-prem soon. Have feedback, or just want to share what you’re building? Join us in the community forums, or reach out to workflows@retool.com. We’re eager to see what you work on next!