If you are working with data in Databricks, you may have encountered the need to analyze, visualize, and share your results with others. That's where Databricks notebooks come into play. But notebooks can turn static, requiring changes to analyze new scenarios. This is exactly where Databricks widgets comes in. Databricks widgets are input elements that allow you to add parameters to your notebooks and dashboards. You can use ‘em to make your Databricks notebooks more interactive and user-friendly.
In this article, we’ll cover everything you need to know about Databricks widgets—from what they are, the different types available, how to create ‘em, real-world examples—and so much more!
What Are Widgets in Databricks?
Databricks widgets are interactive input components that can be added to notebooks to parameterize them and make them more reusable. Widgets enable the passing of arguments and variables into notebooks in a simple way, eliminating the need to hardcode values.
Widgets provide levers and knobs for users to control notebook execution. You can create widgets to select values, lists, dates, files, and much more. These widgets then become arguments that can be referenced throughout your notebook code using programming languages such as Python, R, Scala, or SQL.
TL;DR: Databricks widgets allow you to parameterize your notebooks by creating input widgets that can be adjusted to pass different values into the same notebook code.
Use widgets when you need to:
- Iteratively develop a notebook by re-running it with different parameters
- Build an interactive dashboard that allows changing query parameters on the fly
- Explore the results of a query by adjusting filters and visualizations
The interactive nature of widgets makes them ideal for rapid analysis and dashboard creation.
Benefits of Using Databricks Widgets
Databricks widgets provide several key benefits for building robust and reusable notebooks and dashboards. Here are some of the major benefits of using Databricks widgets:
- Databricks widgets make notebooks highly reusable by letting users change inputs dynamically at runtime instead of having hardcoded values, which allows the same notebook to be used repeatedly with different configurations.
- They avoid having to duplicate notebooks entirely just to tweak some parameters or settings. Users can instead be pointed to an existing parameterized notebook and allowed to interactively play with the widget inputs. This saves an enormous amount of repetitive work.
- Widgets also enable really engaging interactive dashboards to be built where changing the inputs visually updates charts and graphs in real time. Users have full control to flexibly slice and dice the underlying data through an intuitive interface.
- Collaboration is streamlined since teams can work on analyses together through a shared interface of widgets. Ideas can be thrown around and explored interactively by adjusting inputs, similar to virtual whiteboarding.
- Code quality and organization are improved by defining all parameterization in widgets rather than copy-pasting values throughout notebooks. The logic flow is cleaner and easier to understand.
TL;DR: Databricks widgets bring parameterized interactivity to notebooks and dashboards, supporting reusability, collaboration, and simplified workflow.
What Are the Types of Databricks Widgets?
There are 4 main types of built-in Databricks widgets available: text, dropdown, combo box, and multi-select.
- Text Input: Allows capturing single or multi-line text input. It is useful for simple filtering or parameters.
- Dropdown: Provides a dropdown menu to select from a list of options. It is useful for predefined categories and options.
- Combo box: Combines a text box with a drop-down. Users can either type a custom value or select an option.
- Multiselect: Displays a list allowing the selection of multiple options via checkboxes. It is useful for flexible filters and segments.
These cover the most common use cases and data types for passing parameters to notebooks. You can combine multiple widgets together or iterate as needed.
Databricks widget drop-down menus and text input fields appear below the notebook toolbar.
Now that we have covered the basic definitions and use cases of Databricks widgets, let's move on to the next section where we will delve into how to create Databricks widgets.
Step-by-Step Guide to Create Databricks Widgets
There are two ways to create Databricks widgets in Databricks notebooks—using the interface or programmatically via code. Let's go through both options step-by-step:
Step-by-Step Guide to Create Databricks Widgets via UI
The easiest and most common way to add Datatbricks widgets is through the notebook UI. Follow these steps:
Step 1—Open Your Databricks Notebook
Step 2—Adding Widgets
In your notebook, select Edit ➤ Add Widget from the top toolbar menu
Step 2—Filling Widget Details
In the Add Widget dialog box, fill in the details:
- Parameter Name: The parameter name used to reference the widget value in code.
- Widget Label: Optional label text to display above the widget. It is used for user clarity.
- Widget Type: Select from Text, Dropdown, Combobox, Multiselect, DatePicker, etc.
- Parameter Type: Data type of the parameter value, e.g. string or number.
- Default Parameter Value: Optional default value for the widget.
- Choices: For Dropdown, Multiselect, Combobox, enter the list of options here.
Step 3—Configuring Widgets in a Notebook
The widget will appear in your notebook. You can now reference it by the widget name in your code. The widget label is optional UI text to make the widget clear to users.
Step 4—Editing and Removing Databricks Widgets
You can use the kebab menu to edit or remove widgets after creating them.
That’s it! Now, let's move on to the next section, where we will learn how to create Databricks widgets programmatically via code instead of using the UI.
Step-by-Step Guide to Create Databricks Programmatically via Code
Another way to create Databricks widgets is to use the API. You can use the API to create text, dropdown, combo box, and multi-select widgets in Python, Scala, R, and SQL notebooks. Databricks widget API is a set of methods that you can use to create, access, and manipulate widgets in your notebooks. The API is available in the Databricks Utilities (dbutils) reference interface, which is a collection of utilities that you can use to interact with the Databricks environment.
Note: Databricks widget API is the same across Scala, Python, and R. While the SQL widget API is slightly different, but it provides the same functionality.
The widget API consists of the following methods:
- dbutils.widgets.text(): Creates a text widget.
- dbutils.widgets.dropdown(): Creates a dropdown widget.
- dbutils.widgets.combobox(): Creates a combo box widget.
- dbutils.widgets.multiselect(): Creates a multi-select widget.
- dbutils.widgets.get(): Gets the widget value as a string.
- dbutils.widgets.getArgument(): Gets the widget value as an argument.
- dbutils.widgets.remove(): Removes a widget.
- dbutils.widgets.removeAll(): Removes all widgets.
- dbutils.widgets.help(): Shows the detailed API documentation for each method.
Here are the steps to create Databricks widgets:
Step 1—Creating Databricks Widgets
To create a widget programmatically, you need to use one of the widget creation methods: dbutils.widgets.text(), dbutils.widgets.dropdown(), dbutils.widgets.combobox(), or dbutils.widgets.multiselect(). Each method takes different arguments, depending on the type of widget and the language of the notebook. You can use the dbutils.widgets.help() method to see the detailed documentation for each method.
For example, if you want to retrieve the details of each widget, you can use the following code:
%python
# or %Scala or %r
dbutils.widgets.help("text")
dbutils.widgets.help("dropdown")
dbutils.widgets.help("combobox")
dbutils.widgets.help("multiselect")
Here is how you can create Databricks widgets in different languages:
Creating Databricks Widgets—Using SQL
Creating Databricks widgets using SQL is a bit different than using Python, Scala, and R. Here are examples of creating common widget types using SQL:
1) Creating Simple Text Widgets
You can create a Text widget that allows free-form text input using the CREATE WIDGET TEXT statement.
For example:
CREATE WIDGET TEXT address DEFAULT "USA"
As you can see, this defines a Text widget called "address" with a default value of "USA".
2) Creating Simple Dropdown Widgets
A Dropdown widget presents a drop-down selector. Define it using CREATE WIDGET DROPDOWN by passing a SELECT query for the choices:
CREATE WIDGET DROPDOWN country DEFAULT "USA" CHOICES SELECT * FROM (VALUES ("India"), ("USA"), ("China"), ("UAE"), ("Malaysia"));
3) Creating Simple Combobox Widgets
The Combobox is like a Dropdown, but allows text input as well. The syntax is similar:
CREATE WIDGET COMBOBOX combobox_widget DEFAULT "0" CHOICES SELECT * FROM (VALUES ("0"), ("1"), ("2"), ("3"), ("4"));
4) Creating Simple Multi Select Widgets
To allow multiple selections, use CREATE WIDGET MULTISELECT. Pass the choices query like a Dropdown:
CREATE WIDGET MULTISELECT multiselect_widget DEFAULT "Choice 1" CHOICES SELECT * FROM (VALUES ("Choice 1"), ("Choice 2"), ("Choice 3"), ("Choice 4"), ("Choice 5"));
Creating Databricks Widgets—Using Python
Creating Databricks widgets in Python is a bit different than in SQL. To create Databricks widgets in Python, you need to use the %python magic command and the dbutils.widgets methods. Here are some examples of how to create widgets using Python:
1) Creating Simple Text Widgets
Text widgets allow the user to enter free-form text. You can use dbutils.widgets.text() and pass the widget name, default value, and label. The widget name is the name you use to reference the widget in your code. The default value is the value that the widget will have when the notebook is first run or when the widget is reset. The label is the name that will appear above the widget in the UI. For example:
%python
dbutils.widgets.text("Name", "Chaos Genius", "Python Text Widget")
As you can see, this creates a text widget named “Name” with the default value “Chaos Genius” and the label “Python Text Widget”.
2) Creating Simple Dropdown Widgets
Dropdown widget allows the user to select a value from a list of provided values. To create a dropdown widget, you need to use the dbutils.widgets.dropdown() method. This method takes four arguments: the widget name, the default value, the list of choices, and the label. The widget name, the default value, and the label are the same as for the text widget. The list of choices is a list of strings that the user can choose from. For example:
%python
dbutils.widgets.dropdown("Python Dropdown Widget", "0", ["0", "1", "2", "3"]);
As you can see, this creates a dropdown widget named “Python Dropdown Widget” with the default value “0” and the list of choices “0”, “1”, “2”, and “3”.
3) Creating Simple Combobox Widgets
Combobox widgets combine a text input with a dropdown. The user can select from options or enter custom text. You use dbutils.widgets.combobox() and pass the widget name, default value, options list, editable flag, and label. The widget name, the default value, the list of choices, and the label are the same as for the dropdown widget. The editable flag is a boolean value that indicates whether the user can enter a custom value or not. For example:
%python
dbutils.widgets.combobox("Python Combobox Widget", "Other", ["Test 1", "Test 2", "Test 3"])
As you can see, this creates a combobox widget named “Python Combobox Widget” with the default value “Other” and the list of choices “Test 1”, “Test 2”, and “Test 3”, and allows the user to enter a custom value.
4) Creating Simple Multi-Select Widgets
Multiselect widgets allow selecting multiple values from options. You use dbutils.widgets.multiselect() and pass the widget name, default values, options list, and label. The default can be comma-separated selections. The widget name, the default value, and the label are the same as for the dropdown widget. The list of choices is a list of strings that the user can choose from. The default value is a comma-separated string of the selected choices. For example:
%python
dbutils.widgets.multiselect("Python Multiselect Widget", "Choice 1", ["Choice 1", "Choice 2", "Choice 3"])
You can see that this creates a multiselect widget named “Python Multiselect Widget” with the default value “Choice 1” and the list of choices “Choice 1”, “Choice 2”, and “Choice 3”.
This is how your widget should appear.
Creating Databricks Widgets—Using Scala
To create Databricks widgets in Scala, you need to use the %scala magic command and the dbutils.widgets method. The syntax and arguments for creating widgets are the same as for Python, except that the list of choices is a sequence of strings instead of a list of strings. Here are some examples of how to create widgets using Scala:
1) Creating Simple Text Widgets
To create Text widgets in Scala, you can use dbutils.widgets.text() and pass the widget name, default value, and label. It's the same as in Python.
%scala
dbutils.widgets.text("Name", "Chaos Genius", "Scala Text Widget")
As you can see, this creates a text widget named “Name” with the default value “Chaos Genius” and the label “Scala Text Widget”.
2) Creating Simple Dropdown Widgets
To create Dropdown widgets in Scala, you can use dbutils.widgets.dropdown() and pass the widget name, default value, options sequence, and label.
%scala
dbutils.widgets.dropdown("Scala Dropdown Widget", "0", Seq("0", "1", "2", "3"))
As you can see, this creates a dropdown widget named “Scala Dropdown Widget” with the default value “0” and the list of choices “0”, “1”, “2”, and “3”.
3) Creating Simple Combobox Widgets
To create Combobox widgets in Scala, you can use dbutils.widgets.combobox() and pass the widget name, default value, options sequence, editable flag, and label.
%scala
dbutils.widgets.combobox("Scala Combobox Widget", "Other", Seq("Test 1", "Test 2", "Test 3"))
As you can see, this creates a combobox widget named “Scala Combobox Widget” with the default value “Other” and the list of choices “Test 1”, “Test 2”, and “Test 3”, and allows the user to enter a custom value.
4) Creating Simple Multi-Select Widgets
To create Multiselect widgets in Scala, you can use dbutils.widgets.multiselect() and pass the widget name, default values, options sequence, and label. The default can be comma-separated.
%scala
dbutils.widgets.multiselect("Scala Multiselect Widget", "Choice 1", Seq("Choice 1", "Choice 2", "Choice 3"))
As you can see, this creates a multi-select widget named “Scala Multiselect Widget” with the default value “Choice 1” and the list of choices “Choice 1”, “Choice 2”, and “Choice 3”.
This is how your widget should appear.
Creating Databricks Widgets—Using R
To create Databricks widgets in Scala, you need to use the %r magic command and the dbutils.widgets methods. The syntax and arguments for creating widgets are the same as for Python and Scala, except that you need to combine a list of strings using list() instead of listing them out in square brackets like in Python.
1) Creating Simple Text Widgets
To create Text widgets in R, you can use dbutils.widgets.text() and pass the widget name, default value, and label. It's the same as in Scala.
%r
dbutils.widgets.text("Name", "Chaos Genius", "R Text Widget")
2) Creating Simple Dropdown Widgets
To create Dropdown widgets in R, you can use dbutils.widgets.dropdown() and pass the widget name, default value, and options list created using as.list(), and label.
%r
dbutils.widgets.dropdown("R Dropdown Widget", "0", as.list(c("0", "1", "2", "3")))
As you can see, this creates a dropdown widget named “R Dropdown Widget” with the default value “0” and the list of choices “0”, “1”, “2”, and “3”.
3) Creating Simple Combobox Widgets
To create Combobox widgets in R, you can use dbutils.widgets.combobox() and pass the widget name, default value, options list, editable flag set to TRUE, and label. Initialize it as:
%r
dbutils.widgets.combobox("R Combobox Widget", "Other", as.list(c("Test 1", "Test 2", "Test 3"), TRUE))
As you can see, this creates a combobox widget named “R Combobox Widget” with the default value “Other” and the list of choices “Test 1”, “Test 2”, and “Test 3”, and allows the user to enter a custom value.
4) Creating Simple Multi-Select Widgets
To create Multiselect widgets in R, you can use dbutils.widgets.multiselect() and pass the widget name, default values, options sequence, and label. The default can be comma-separated.
%r
dbutils.widgets.multiselect("R Multiselect Widget", "Choice 1, Choice 2", as.list(c("Choice 1", "Choice 2", "Choice 3")))
As you can see, this creates a multi-select widget named “R Multiselect Widget” with the default value “Choice 1” and the list of choices “Choice 1”, “Choice 2”, and “Choice 3”.
This is how your widget should appear.
Configuring Databricks Widget Settings in Notebooks
Follow these steps to configure the behavior and layout of Databricks widgets in a notebook:
Click the settings ⚙️ icon on the far right of the Widgets panel.
This opens the Widget Panel Settings dialog box. Under "On Widget changes", choose the widget's execution behavior:
- Select "Run Notebook" to rerun the entire notebook when a new value is picked in a widget.
- Select "Run Accessed Commands" to only rerun cells that retrieve values for that widget when its value changes. This is the default setting. Notebook cells will not rerun.
- Select "Do Nothing" if you don't want anything to rerun when a widget's value is changed.
Pinning the Widgets
Click on the “Pin” 📌 option to keep the Widgets panel pinned to the top as you scroll through the notebook. Unclick to allow the panel to scroll off the screen.
Configuring Databricks Widgets Layout
To configure the Databricks widgets layout, you can do so by clicking the layout icon on the far right of the Widgets panel. Choose and drag it according to your liking inside that panel.
Click ✔️ to save the widget layout settings or click ❌ to cancel the layout.
To reset the Databricks widgets layout, you can do so by clicking the settings ⚙️ icon on the far right of the Widgets panel. This opens the Widget Panel Settings dialog box. Now click on “Reset Layout” to completely reset the layout of the widgets.
Real-World Examples of Using Databricks Widgets
Now, let's jump into the main essence of the article, where we will create a real-world example and scenario using Databricks widgets. This will give you a complete understanding of how to utilize the widgets properly. In this section, we will load the Squirrel Census data into Databricks and create various types of widgets to filter the data. We will then build charts and dashboards to interactively visualize the filtered data. So, without any further ado, let's dive right into it.
Step 1—Loading Squirrel Census Dataset into Databricks
First, let's begin by loading the dataset into Databricks. Start by downloading the Squirrel Census dataset from this site. Once you have downloaded the dataset, navigate to "Data ingestion" in Databricks and load the dataset.
Our dataset has been successfully created. Now, let's create a temporary view/table to store and query that dataset. To do this, execute the query mentioned below:
CREATE OR REPLACE TEMPORARY VIEW squirrel_data
USING CSV
OPTIONS (path "/FileStore/tables/squirrel_data-1.csv", header "true", mode "FAILFAST");
If you query the data, you should be able to see that all of our data has been loaded.
SELECT * FROM squirrel_data;
Let's proceed to the next step and begin the actual creation of Databricks widgets using SQL, Python, Spark, and R.
Step 2—Create Databricks Widgets
In this step, we will use SQL, Python, Scala, and R to create different types of Databricks widgets. It's important to note that for Python, Scala, and R, the steps are similar to those discussed earlier in the article.
However, note that the functionality of our Databricks widgets will be the same across all languages.
Using SQL:
1) Creating Text Databricks widgets—Using SQL
CREATE WIDGET TEXT text_widget_via_sql DEFAULT "01";
The CREATE WIDGET TEXT statement creates a text widget called text_widget_via_sql with a default value of "01".
2) Creating Dropdown Databricks widgets—Using SQL
CREATE WIDGET DROPDOWN dropdown_widget_via_sql DEFAULT "Gray" CHOICES (VALUES 'White', 'Cinnamon', 'Gray', 'Black', "");
The CREATE WIDGET DROPDOWN statement creates a dropdown widget called dropdown_widget_via_sql with default "Gray". The CHOICES list provides the dropdown options.
3) Creating Dropdown Databricks widgets—Using SQL
CREATE WIDGET COMBOBOX combobox_widget_via_sql DEFAULT "01" CHOICES SELECT DISTINCT `Park ID` FROM squirrel_data;
The CREATE WIDGET COMBOBOX statement creates a combobox widget called combobox_widget_via_sql, defaulting to "01". The CHOICES query the table to populate unique values.
4) Creating Multi Select Databricks widgets—Using SQL
CREATE WIDGET MULTISELECT multiselect_widget_via_sql DEFAULT "Gray" CHOICES (VALUES 'White', 'Cinnamon', 'Black', 'Gray');
The CREATE WIDGET MULTISELECT statement creates a multiselect widget called multiselect_widget_via_sql with "Gray" initially selected. CHOICES defines the checkbox options.
Once the code is executed, the widgets will appear at the top of the notebook.
Using Python:
1) Creating Text Databricks widgets—Using Python
To create a text widget using Python, you can use the dbutils.widgets.text method.
%python
dbutils.widgets.text("text_widget_via_python", "01", "Park ID")
As you can see, this creates a text widget called text_widget_via_python with a default value of "01".
2) Creating Dropdown Databricks widgets—Using Python
To create a dropdown widget using Python, you can use the dbutils.widgets.dropdown method.
%python
dbutils.widgets.dropdown("dropdown_widget_via_python", "Gray", ["Gray", "White", "Cinnamon", "Black"], "Highlights in Fur Color")
As you can see, this creates a dropdown widget called dropdown_widget_via_python with a default value of "Gray".
3) Creating Combobox Databricks widgets—Using Python
To create a combobox widget using Python, you can use the dbutils.widgets.combobox method.
%python
import pandas as pd
df = pd.read_csv("/dbfs/FileStore/tables/squirrel_data-1.csv", header=0, encoding='latin1')
unique_park_ids = df['Park ID'].unique().tolist()
unique_park_ids_str = list(map(str, unique_park_ids))
dbutils.widgets.combobox("combobox_widget_via_python", "1", unique_park_ids_str, "Park ID")
Here a combobox is created by reading a CSV into a Pandas DataFrame, getting unique values from a column, and passing them to dbutils.widgets.combobox to populate the options.
4) Creating Multi Select Databricks widgets—Using Python
To create a multiselect widget using Python, you can use the dbutils.widgets.multiselect method.
%python
dbutils.widgets.multiselect("multiselect_widget_via_python", "Gray", ["Gray", "White", "Cinnamon", "Black"], "Highlights in Fur Color")
As you can see, this creates a multiselect widget called multiselect_widget_via_python with "Gray" initially selected.
Once the code is executed, the widgets will appear at the top of the notebook.
Using Scala:
Note: Syntax and arguments for creating widgets in Scala are the same as those in Python.
1) Creating Text Databricks widgets—Using Scala
%scala
dbutils.widgets.text("text_widget_via_scala", "01", "Park ID")
As you can see, this creates a text widget called text_widget_via_scala with a default of "01".
2) Creating Dropdown Databricks widgets—Using Scala
%scala
dbutils.widgets.dropdown("dropdown_widget_via_scala", "Gray", Seq("Gray", "White", "Cinnamon", "Black"), "Highlights in Fur Color")
You can see that this code generates a dropdown widget named dropdown_widget_via_scala with a default value of "Gray". Notice that in Scala, we create a sequence using Seq().
3) Creating Combobox Databricks widgets—Using Scala
%scala
dbutils.widgets.combobox("combobox_widget_via_scala", "01", spark.sql("SELECT DISTINCT `Park ID` FROM squirrel_data").collect().map(_.getString(0)), "Park ID")
Here a SQL query is used to get distinct values for a combobox. The collect/map transforms the data into strings for the options.
4) Creating Multi Select Databricks widgets—Using Scala
%scala
dbutils.widgets.multiselect("multiselect_widget_via_scala", "Gray", Seq("Gray", "White", "Cinnamon", "Black"), "Highlights in Fur Color")
This code creates a multiselect widget called multiselect_widget_via_scala with "Gray" initially selected.
Once the code is executed, the widgets will appear at the top of the notebook.
Using R:
Note: Syntax and arguments for creating widgets in R are the same as those in Python.
1) Creating Text Databricks widgets—Using R
%r
dbutils.widgets.text("text_widget_via_r", "01", "Park ID")
As you can see, the dbutils.widgets.text method creates a text widget called text_widget_via_r with a default value of "01".
2) Creating Dropdown Databricks widgets—Using R
%r
dbutils.widgets.dropdown("dropdown_widget_via_r", "Gray", as.list(c("Gray", "White", "Cinnamon", "Black")), "Highlights in Fur Color")
You can see, that the dbutils.widgets.dropdown method creates a dropdown widget called dropdown_widget_via_r with a default value of "Gray". The as.list() function converts the choices to a list.
3) Creating Combobox Databricks widgets—Using R
%r
if (!requireNamespace("SparkR", quietly = TRUE)) {
install.packages("SparkR")
}
library(SparkR)
# Retrieve the result as a DataFrame
result <- sql("SELECT DISTINCT `Park ID` FROM squirrel_data")
# Convert DataFrame to a regular R dataframe
result_df <- collect(result)
# Extract column as a vector
park_ids <- result_df$`Park ID`
# Convert vector to a list
park_ids_list <- as.list(park_ids)
# Use the list in the combobox widget
dbutils.widgets.combobox("combobox_widget_via_r", "01", park_ids_list, "Park ID")
In this example, a SQL query is used to retrieve distinct values for the combobox. The result is collected and converted to a dataframe, then a vector, and finally a list to pass to the widget.
4) Creating Multi Select Databricks widgets—Using R
To create a multiselect widget using R, you can use the dbutils.widgets.multiselect method.
%r
dbutils.widgets.multiselect("multiselect_widget_via_r", "Gray", as.list(c("Gray", "White", "Cinnamon", "Black")), "Highlights in Fur Color")
Here the dbutils.widgets.multiselect method creates a multiselect widget called multiselect_widget_via_r with "Gray" initially selected. The choices are converted to a list with as.list().
Once the code is executed, the widgets will appear at the top of the notebook.
Step 3—Fetching Widget Values
Once you've created the Databricks widget, you can now retrieve data or values for the widget.
Note: You can hover over the widget name, and a tooltip will appear, guiding how to proceed.
Fetching Databricks widgets value in SQL
To retrieve the widget value using SQL, simply enclose the widget name within ${widget_name} syntax, like this:
SELECT *
FROM squirrel_data
WHERE `Park ID` = ${text_widget_via_sql};
As you can see, this will substitute the value selected in the text_widget_via_sql widget into the SQL query dynamically.
You can also fetch multiple widget values at once:
SELECT `Park ID`, `Area Name`, `Park Name`, `Primary Fur Color`
FROM squirrel_data
WHERE `Park ID` = ${text_widget_via_sql}
AND `Area Name` = ${combobox_widget_via_sql};
Fetching Databricks widgets value in Python
In Python notebooks, we can fetch the Databricks widgets values using the dbutils.widget.get(“widget_name”):
%python
# Fetch widget values
park_id = dbutils.widgets.get("text_widget_via_python")
fur_color = dbutils.widgets.get("multiselect_widget_via_python")
# Read DataFrame
df = spark.read.format("csv").option("header", "true").load("/FileStore/tables/squirrel_data-1.csv")
# Filter DataFrame based on widgets
df_filtered = df.filter((df["Park ID"] == park_id) & (df["Highlights in Fur Color"] == fur_color))
display(df_filtered)
Fetching Databricks widgets value in Scala
In Scala notebooks use the same dbutils.widgets.get() method:
%scala
val park_id = dbutils.widgets.get("text_widget_via_scala")
val fur_color = dbutils.widgets.get("multiselect_widget_via_scala")
val df = spark.read.format("csv").option("header", "true").load("/FileStore/tables/squirrel_data-1.csv")
val df_filtered = df.filter((df("Park ID") === park_id) && (df("Highlights in Fur Color") === fur_color))
display(df_filtered)This will fetch the widget values into Scala variables to use in Spark operations.
Step 4—Using Databricks Widgets as Filters for the Dashboard
Let's begin by generating a Databricks widget and exploring its application as a filter for the Databricks dashboard. To initiate the process, create a chart using the built-in tool provided by Databricks. Access the dashboard by navigating to the top right corner of the cell and selecting the bar/graph icon. Click on it, and a new button labeled "Add to new dashboard" will appear. Simply click on this button to create a new dashboard.
Upon accessing the dashboard, you'll notice its interface. You can effortlessly utilize the widgets as knobs and levers to configure the data as needed.
Step 5—Deleting Databricks Widgets
After you have finished using the Databricks widgets, you may want to delete them to free up some space and avoid cluttering your notebook or dashboard. Deleting a widget will remove it from the UI and also from the widget registry, which stores the widget names and values.
There are different ways to delete Databricks widgets:
1) Removing only one Databricks widgets
To remove only one Databricks widget using SQL, you can use the REMOVE WIDGET command.
REMOVE WIDGET dropdown_widget_via_scala
2) Remove all Databricks widgets
To remove all Databricks widgets, you can use the following code:
%python
dbutils.widgets.removeAll();
Now that you have a clear understanding of Databricks widgets and how to use them for parameterization in Databricks, you can start using them effortlessly. Let's move on to the next section where we will discuss the limitations and drawbacks of using Databricks Widgets.
What are the Limitations & Drawbacks of Using Databricks Widgets?
Databricks widgets have some limitations and drawbacks that you have to keep in mind. They are:
- There can only be a maximum of 512 widgets per notebook; exceeding this limit may lead to issues.
- Widget names are restricted to 1024 characters.
- The label for each widget can have a maximum of 2048 characters.
- Text input widgets have a limit of 2048 characters, potentially truncating long strings or code snippets.
- Dropdowns, comboboxes, and multi-select widgets have a cap of 1024 options, and very large lists can impact performance.
- Creating and rendering numerous widgets on each run may slow down execution speed.
- An excess of dropdowns or large inputs can affect the user interface, leading to performance issues.
- Resource-intensive notebooks may experience further slowdowns due to the presence of many widgets.
- Since widgets create stateful and customizable notebooks, sharing and explaining them becomes more challenging.
While the performance impact is generally not severe, it is advisable to use Databricks widgets cautiously, especially in already complex notebooks.
Conclusion
And that’s a wrap! Databricks widgets are a really cool and powerful tool that lets you customize your notebooks and dashboards. Being able to tweak values on the fly means your notebooks can adapt to new scenarios without rewriting code. You can even collaborate more easily by letting others play around with different inputs.
In this article, we have covered:
- What Are Widgets in Databricks?
- Benefits of Using Databricks Widgets
- What Are the Types of Databricks Widgets?
- Step by Step Guide to Create Databricks Widgets via UI
- Step by Step Guide to Create Databricks Widgets Programmatically via Code
- Configuring Databricks Widget Settings
- Real-World Examples of Using Databricks Widgets
- Limitations & Drawbacks of Using Databricks Widgets
…and more!
Databricks widgets are like adding customizable knobs and levers to your data analysis, giving you finer control over the insights you can generate.
FAQs
What are Databricks widgets?
Databricks widgets are interactive input components that can be added to notebooks to parameterize them and make them more reusable.
What are the different types of Databricks widgets?
There are 4 different types of Databricks widgets: text input, dropdown, combo box, and multi-select widgets.
How do you create Databricks widgets in a notebook?
You can create widgets through the notebook UI or programmatically via Python, Scala, R, or SQL code using the Databricks widget API.
How do you configure Databricks widget settings?
You can configure execution behavior and layout using the settings icon in the widget panel. Options include pinning, resetting layout, and setting rerun logic.
How do you reference a Databricks widget value in code?
Enclosing the widget name in ${widget_name} syntax for SQL, or using dbutils.widgets.get("widget_name") in Python, Scala, and R.
How do you delete Databricks widgets when no longer needed?
You can use the REMOVE WIDGET command in SQL or dbutils.widgets.remove() method in Python, Scala, and R. dbutils.widgets.removeAll() deletes all widgets.
What is the maximum number of widgets allowed per notebook?
There is a limit of 512 widgets per notebook. Exceeding this can lead to issues.
What are the character limits for widget names and labels?
Widget names have a 1024-character limit. Labels have a 2048-character limit.
What are the options limits for dropdown and multiselect widgets?
Dropdown, combobox, and multiselect widgets can have up to 1024 options.
What is the text limit for text input widgets?
Text input widgets have a limit of 2048 characters. Longer strings get truncated.
What are some limitations of using many Databricks widgets?
Too many widgets can slow down execution, affect UI performance, and make notebooks hard to understand. Use carefully.
Are Databricks widgets supported in SQL, Python, R, and Scala notebooks?
Yes, the widget API is available across SQL, Python, R, and Scala notebooks. With minor syntax differences.
Do Databricks widgets work in notebook-backed dashboards?
Yes, widgets enable building parameterized dashboards using notebooks as data sources.
Can Databricks widgets accept file uploads or selections?
No, file uploads are not supported directly. But you can build UIs with 3rd party libraries to enable this.
Are there limitations when sharing notebooks with widgets?
Notebooks with widgets may not work if the recipient lacks cluster access to rerun them.
Can Databricks widgets be used in production workflows?
Yes, but they should be used carefully. Test thoroughly and minimize usage for performance.
