Merge branch 'main' into fp/applicatives

.github/workflows/typos.yaml

@@ -13,3 +13,5 @@ jobs:
         uses: styfle/[email protected]
       - uses: actions/checkout@v4
       - uses: crate-ci/[email protected]
+        with:
+          config: .typos.toml

.typos.toml

@@ -0,0 +1,22 @@
+[default]
+extend-ignore-re = [
+    # LaTeX math expressions
+    "\\\\\\[.*?\\\\\\]",
+    "\\\\\\(.*?\\\\\\)",
+    "\\$\\$.*?\\$\\$",
+    "\\$.*?\\$",
+    # LaTeX commands
+    "\\\\[a-zA-Z]+\\{.*?\\}",
+    "\\\\[a-zA-Z]+",
+    # LaTeX subscripts and superscripts
+    "_\\{.*?\\}",
+    "\\^\\{.*?\\}"
+]
+
+# Words to explicitly accept
+[default.extend-words]
+pn = "pn"
+
+# You can also exclude specific files or directories if needed
+# [files]
+# extend-exclude = ["*.tex", "docs/*.md"] 

functional_programming/README.md

@@ -38,7 +38,7 @@ uvx marimo edit <URL>
 For example, run the `Functor` tutorial with
 
 ```bash
-uvx marimo edit https://github.com/marimo-team/learn/blob/main/Functional_programming/05_functors.py
+uvx marimo edit https://github.com/marimo-team/learn/blob/main/functional_programming/05_functors.py
 ```
 
 You can also open notebooks in our online playground by appending `marimo.app/`
@@ -50,10 +50,9 @@ to a notebook's URL:
 Check [here](https://github.com/marimo-team/learn/issues/51) for current series
 structure.
 
-| Notebook | Title | Description | Key Concepts | Prerequisites |
-|----------|-------|-------------|--------------|---------------|
-| [05. Functors](https://github.com/marimo-team/learn/blob/main/Functional_programming/05_functors.py) | Category and Functors | Learn why `len` is a _Functor_ from `list concatenation` to `integer addition`, how to _lift_ an ordinary function into a _computation context_, and how to write an _adapter_ between two categories. | Categories, Functors, Function lifting, Context mapping | Basic Python, Functions |
-| [06. Applicatives](https://github.com/marimo-team/learn/blob/main/Functional_programming/06_applicatives.py) | Applicative programming with effects | Learn how to apply functions within a context, combining multiple effects in a pure way. Learn about the `pure` and `apply` operations that make applicatives powerful for handling multiple computations. | Applicative Functors, Pure, Apply, Effectful programming | Functors |
+| Notebook                                                                                                          | Description                                                                                                                                                                                            | References                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
+| ----------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| [05. Category and Functors](https://github.com/marimo-team/learn/blob/main/functional_programming/05_functors.py) | Learn why `len` is a _Functor_ from `list concatenation` to `integer addition`, how to _lift_ an ordinary function into a _computation context_, and how to write an _adapter_ between two categories. | - [The Trivial Monad](http://blog.sigfpe.com/2007/04/trivial-monad.html) <br> - [Haskellwiki. Category Theory](https://en.wikibooks.org/wiki/Haskell/Category_theory) <br> - [Haskellforall. The Category Design Pattern](https://www.haskellforall.com/2012/08/the-category-design-pattern.html) <br> - [Haskellforall. The Functor Design Pattern](https://www.haskellforall.com/2012/09/the-functor-design-pattern.html) <br> - [Haskellwiki. Functor](https://wiki.haskell.org/index.php?title=Functor) <br> - [Haskellwiki. Typeclassopedia#Functor](https://wiki.haskell.org/index.php?title=Typeclassopedia#Functor) <br> - [Haskellwiki. Typeclassopedia#Category](https://wiki.haskell.org/index.php?title=Typeclassopedia#Category) |
 
 **Authors.**
 

polars/08_working_with_columns.py

@@ -0,0 +1,605 @@
+# /// script
+# requires-python = ">=3.11"
+# dependencies = [
+#     "polars==1.18.0",
+#     "marimo",
+# ]
+# ///
+
+import marimo
+
+__generated_with = "0.12.0"
+app = marimo.App(width="medium")
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        # Working with Columns
+
+        Author: [Deb Debnath](https://github.com/debajyotid2)
+
+        **Note**: The following tutorial has been adapted from the Polars [documentation](https://docs.pola.rs/user-guide/expressions/expression-expansion).
+        """
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        ## Expressions
+
+        Data transformations are sometimes complicated, or involve massive computations which are time-consuming. You can make a small version of the dataset with the schema you are trying to work your transformation into. But there is a better way to do it in Polars.
+
+        A Polars expression is a lazy representation of a data transformation. "Lazy" means that the transformation is not eagerly (immediately) executed. 
+
+        Expressions are modular and flexible. They can be composed to build more complex expressions. For example, to calculate speed from distance and time, you can have an expression as:
+        """
+    )
+    return
+
+
+@app.cell
+def _(pl):
+    speed_expr = pl.col("distance") / (pl.col("time"))
+    speed_expr
+    return (speed_expr,)
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        ## Expression expansion
+
+        Expression expansion lets you write a single expression that can expand to multiple different expressions. So rather than repeatedly defining separate expressions, you can avoid redundancy while adhering to clean code principles (Do not Repeat Yourself - [DRY](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself)). Since expressions are reusable, they aid in writing concise code.
+        """
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md("""For the examples in this notebook, we will use a sliver of the *AI4I 2020 Predictive Maintenance Dataset*. This dataset comprises of measurements taken from sensors in industrial machinery undergoing preventive maintenance checks - basically being tested for failure conditions.""")
+    return
+
+
+@app.cell
+def _(StringIO, pl):
+    data_csv = """
+    Product ID,Type,Air temperature,Process temperature,Rotational speed,Tool wear,Machine failure,TWF,HDF,PWF,OSF,RNF
+    L51172,L,302.3,311.3,1614,129,0,0,1,0,0,0
+    M22586,M,300.8,311.9,1761,113,1,0,0,0,1,0
+    L51639,L,302.6,310.4,1743,191,0,1,0,0,0,1
+    L50250,L,300,309.1,1631,110,0,0,0,1,0,0
+    M20109,M,303.4,312.9,1422,63,1,0,0,0,0,0
+    """
+
+    data = pl.read_csv(StringIO(data_csv))
+    data
+    return data, data_csv
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        ## Function `col`
+
+        The function `col` is used to refer to one column of a dataframe. It is one of the fundamental building blocks of expressions in Polars. `col` is also really handy in expression expansion.
+        """
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        ### Explicit expansion by column name
+
+        The simplest form of expression expansion happens when you provide multiple column names to the function `col`.
+
+        Say you wish to convert all temperature values in deg. Kelvin (K) to deg. Fahrenheit (F). One way to do this would be to define individual expressions for each column as follows:
+        """
+    )
+    return
+
+
+@app.cell
+def _(data, pl):
+    exprs = [
+        ((pl.col("Air temperature") - 273.15) * 1.8 + 32).round(2),
+        ((pl.col("Process temperature") - 273.15) * 1.8 + 32).round(2)
+    ]
+
+    result = data.with_columns(exprs)
+    result
+    return exprs, result
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""Expression expansion can reduce this verbosity when you list the column names you want the expression to expand to inside the `col` function. The result is the same as before.""")
+    return
+
+
+@app.cell
+def _(data, pl, result):
+    result_2 = data.with_columns(
+        (
+            (pl.col(
+                "Air temperature",
+                "Process temperature"
+            )
+            - 273.15) * 1.8 + 32
+        ).round(2)
+    )
+    result_2.equals(result)
+    return (result_2,)
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""In this case, the expression that does the temperature conversion is expanded to a list of two expressions. The expansion of the expression is predictable and intuitive.""")
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        ### Expansion by data type
+
+        Can we do better than explicitly writing the names of every columns we want transformed? Yes.
+
+        If you provide data types instead of column names, the expression is expanded to all columns that match one of the data types provided.
+
+        The example below performs the exact same computation as before:
+        """
+    )
+    return
+
+
+@app.cell
+def _(data, pl, result):
+    result_3 = data.with_columns(((pl.col(pl.Float64) - 273.15) * 1.8 + 32).round(2))
+    result_3.equals(result)
+    return (result_3,)
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        However, you should be careful to ensure that the transformation is only applied to the columns you want. For ensuring this it is important to know the schema of the data beforehand. 
+
+        `col` accepts multiple data types in case the columns you need have more than one data type.
+        """
+    )
+    return
+
+
+@app.cell
+def _(data, pl, result):
+    result_4 = data.with_columns(
+        (
+            (pl.col(
+                pl.Float32,
+                pl.Float64,
+            )
+             - 273.15) * 1.8 + 32
+        ).round(2)
+    )
+    result.equals(result_4)
+    return (result_4,)
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        ### Expansion by pattern matching
+
+        `col` also accepts regular expressions for selecting columns by pattern matching. Regular expressions start and end with ^ and $, respectively.
+        """
+    )
+    return
+
+
+@app.cell
+def _(data, pl):
+    data.select(pl.col("^.*temperature$"))
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""Regular expressions can be combined with exact column names.""")
+    return
+
+
+@app.cell
+def _(data, pl):
+    data.select(pl.col("^.*temperature$", "Tool wear"))
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""**Note**: You _cannot_ mix strings (exact names, regular expressions) and data types in a `col` function.""")
+    return
+
+
+@app.cell
+def _(data, pl):
+    try:
+        data.select(pl.col("Air temperature", pl.Float64))
+    except TypeError as err:
+        print("TypeError:", err)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        ## Selecting all columns
+
+        To select all columns, you can use the `all` function.
+        """
+    )
+    return
+
+
+@app.cell
+def _(data, pl):
+    result_6 = data.select(pl.all())
+    result_6.equals(data)
+    return (result_6,)
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        ## Excluding columns
+
+        There are scenarios where we might want to exclude specific columns from the ones selected by building expressions, e.g. by the `col` or `all` functions. For this purpose, we use the function `exclude`, which accepts exactly the same types of arguments as `col`:
+        """
+    )
+    return
+
+
+@app.cell
+def _(data, pl):
+    data.select(pl.all().exclude("^.*F$"))
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""`exclude` can also be used after the function `col`:""")
+    return
+
+
+@app.cell
+def _(data, pl):
+    data.select(pl.col(pl.Int64).exclude("^.*F$"))
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        ## Column renaming
+
+        When applying a transformation with an expression to a column, the data in the column gets overwritten with the transformed data. However, this might not be the intended outcome in all situations - ideally you would want to store transformed data in a new column. Applying multiple transformations to the same column at the same time without renaming leads to errors.
+        """
+    )
+    return
+
+
+@app.cell
+def _(data, pl):
+    from polars.exceptions import DuplicateError
+
+    try:
+        data.select(
+            (pl.col("Air temperature") - 273.15) * 1.8 + 32,  # This would be named "Air temperature"...
+            pl.col("Air temperature") - 273.15,  # And so would this.
+        )
+    except DuplicateError as err:
+        print("DuplicateError:", err)
+    return (DuplicateError,)
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        ### Renaming a single column with `alias`
+
+        The function `alias` lets you rename a single column:
+        """
+    )
+    return
+
+
+@app.cell
+def _(data, pl):
+    data.select(
+            ((pl.col("Air temperature") - 273.15) * 1.8 + 32).round(2).alias("Air temperature [F]"),
+            (pl.col("Air temperature") - 273.15).round(2).alias("Air temperature [C]")
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        ### Prefixing and suffixing column names
+
+        As `alias` renames a single column at a time, it cannot be used during expression expansion. If it is sufficient add a static prefix or a static suffix to the existing names, you can use the functions `name.prefix` and `name.suffix` with `col`:
+        """
+    )
+    return
+
+
+@app.cell
+def _(data, pl):
+    data.select(
+        ((pl.col("Air temperature") - 273.15) * 1.8 + 32).round(2).name.prefix("deg F "),
+        (pl.col("Process temperature") - 273.15).round(2).name.suffix(" C"),
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        ### Dynamic name replacement
+
+        If a static prefix/suffix is not enough, use `name.map`. `name.map` requires a function that transforms column names to the desired. The transformation should lead to unique names to avoid `DuplicateError`.
+        """
+    )
+    return
+
+
+@app.cell
+def _(data, pl):
+    # There is also `.name.to_lowercase`, so this usage of `.map` is moot.
+    data.select(pl.col("^.*F$").name.map(str.lower))
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        ## Programmatically generating expressions
+
+        For this example, we will first create four additional columns with the rolling mean temperatures of the two temperature columns. Such transformations are sometimes used to create additional features for machine learning models or data analysis.
+        """
+    )
+    return
+
+
+@app.cell
+def _(data, pl):
+    ext_temp_data = data.with_columns(
+            pl.col("^.*temperature$").rolling_mean(window_size=2).round(2).name.prefix("Rolling mean ")
+    ).select(pl.col("^.*temperature*$"))
+    ext_temp_data
+    return (ext_temp_data,)
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""Now, suppose we want to calculate the difference between the rolling mean and actual temperatures. We cannot use expression expansion here as we want differences between specific columns.""")
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""At first, you may think about using a `for` loop:""")
+    return
+
+
+@app.cell
+def _(ext_temp_data, pl):
+    _result = ext_temp_data
+    for col_name in ["Air", "Process"]:
+        _result = _result.with_columns(
+            (abs(pl.col(f"Rolling mean {col_name} temperature") - pl.col(f"{col_name} temperature")))
+                .round(2).alias(f"Delta {col_name} temperature")
+        )
+    _result
+    return (col_name,)
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""Using a `for` loop is functional, but not scalable, as each expression needs to be defined in an iteration and executed serially. Instead we can use a generator in Python to programmatically create all expressions at once. In conjunction with the `with_columns` context, we can take advantage of parallel execution of computations and query optimization from Polars.""")
+    return
+
+
+@app.cell
+def _(ext_temp_data, pl):
+    def delta_expressions(colnames: list[str]) -> pl.Expr:
+        for col_name in colnames:
+            yield (abs(pl.col(f"Rolling mean {col_name} temperature") - pl.col(f"{col_name} temperature"))
+                            .round(2).alias(f"Delta {col_name} temperature"))
+
+
+    ext_temp_data.with_columns(delta_expressions(["Air", "Process"]))
+    return (delta_expressions,)
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        ## More flexible column selections
+
+        For more flexible column selections, you can use column selectors from `selectors`. Column selectors allow for more expressiveness in the way you specify selections. For example, column selectors can perform the familiar set operations of union, intersection, difference, etc. We can use the union operation with the functions `string` and `ends_with` to select all string columns and the columns whose names end with "`_high`":
+        """
+    )
+    return
+
+
+@app.cell
+def _(data):
+    import polars.selectors as cs
+
+    data.select(cs.string() | cs.ends_with("F"))
+    return (cs,)
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""Likewise, you can pick columns based on the category of the type of data, offering more flexibility than the `col` function. As an example, `cs.numeric` selects numeric data types (including `pl.Float32`, `pl.Float64`, `pl.Int32`, etc.) or `cs.temporal` for all dates, times and similar data types.""")
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        ### Combining selectors with set operations
+
+        Multiple selectors can be combined using set operations and the usual Python operators:
+
+
+        | Operator |       Operation      |
+        |:--------:|:--------------------:|
+        | `A | B`   | Union                |
+        | `A & B`    | Intersection         |
+        | `A - B`    | Difference           |
+        | `A ^ B`    | Symmetric difference |
+        | `~A`       | Complement           |
+
+        For example, to select all failure indicator variables excluding the failure variables due to wear, we can perform a set difference between the column selectors.
+        """
+    )
+    return
+
+
+@app.cell
+def _(cs, data):
+    data.select(cs.contains("F") - cs.contains("W"))
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        ### Resolving operator ambiguity
+
+        Expression functions can be chained on top of selectors:
+        """
+    )
+    return
+
+
+@app.cell
+def _(cs, data, pl):
+    ext_failure_data = data.select(cs.contains("F")).cast(pl.Boolean)
+    ext_failure_data
+    return (ext_failure_data,)
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        However, operators that perform set operations on column selectors operate on both selectors and on expressions. For example, the operator `~` on a selector represents the set operation “complement” and on an expression represents the Boolean operation of negation.
+
+        For instance, if you want to negate the Boolean values in the columns “HDF”, “OSF”, and “RNF”, at first you would think about using the `~` operator with the column selector to choose all failure variables containing "W". Because of the operator ambiguity here, the columns that are not of interest are selected here.
+        """
+    )
+    return
+
+
+@app.cell
+def _(cs, ext_failure_data):
+    ext_failure_data.select((~cs.ends_with("WF")).name.prefix("No"))
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""To resolve the operator ambiguity, we use `as_expr`:""")
+    return
+
+
+@app.cell
+def _(cs, ext_failure_data):
+    ext_failure_data.select((~cs.ends_with("WF").as_expr()).name.prefix("No"))
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        ### Debugging selectors
+
+        The function `cs.is_selector` helps check whether a complex chain of selectors and operators ultimately results in a selector. For example, to resolve any ambiguity with the selector in the last example, we can do:
+        """
+    )
+    return
+
+
+@app.cell
+def _(cs):
+    cs.is_selector(~cs.ends_with("WF").as_expr())
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""Additionally we can use `expand_selector` to see what columns a selector expands into. Note that for this function we need to provide additional context in the form of the dataframe.""")
+    return
+
+
+@app.cell
+def _(cs, ext_failure_data):
+    cs.expand_selector(
+        ext_failure_data,
+        cs.ends_with("WF"),
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        ### References
+
+        1. AI4I 2020 Predictive Maintenance Dataset [Dataset]. (2020). UCI Machine Learning Repository. ([link](https://doi.org/10.24432/C5HS5C)).
+        2. Polars documentation ([link](https://docs.pola.rs/user-guide/expressions/expression-expansion/#more-flexible-column-selections))
+        """
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _():
+    import csv
+    import marimo as mo
+    import polars as pl
+    from io import StringIO
+    return StringIO, csv, mo, pl
+
+
+if __name__ == "__main__":
+    app.run()

polars/09_data_types.py

@@ -0,0 +1,296 @@
+# /// script
+# requires-python = ">=3.11"
+# dependencies = [
+#     "polars==1.18.0",
+#     "marimo",
+# ]
+# ///
+
+import marimo
+
+__generated_with = "0.12.0"
+app = marimo.App(width="medium")
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        # Data Types
+
+        Author: [Deb Debnath](https://github.com/debajyotid2)
+
+        **Note**: The following tutorial has been adapted from the Polars [documentation](https://docs.pola.rs/user-guide/concepts/data-types-and-structures/).
+        """
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        Polars supports a variety of data types that fall broadly under the following categories:
+
+        - Numeric data types: integers and floating point numbers.
+        - Nested data types: lists, structs, and arrays.
+        - Temporal: dates, datetimes, times, and time deltas.
+        - Miscellaneous: strings, binary data, Booleans, categoricals, enums, and objects.
+
+        All types support missing values represented by `null` which is different from `NaN` used in floating point data types. The numeric datatypes in Polars loosely follow the type system of the Rust language, since its core functionalities are built in Rust.
+
+        [Here](https://docs.pola.rs/api/python/stable/reference/datatypes.html) is a full list of all data types Polars supports.
+        """
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        ## Series
+
+        A series is a 1-dimensional data structure that can hold only one data type.
+        """
+    )
+    return
+
+
+@app.cell
+def _(pl):
+    s = pl.Series("emojis", ["😀", "🤣", "🥶", "💀", "🤖"])
+    s
+    return (s,)
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""Unless specified, Polars infers the datatype from the supplied values.""")
+    return
+
+
+@app.cell
+def _(pl):
+    s1 = pl.Series("friends", ["Евгений", "अभिषेक", "秀良", "Federico", "Bob"])
+    s2 = pl.Series("uints", [0x00, 0x01, 0x10, 0x11], dtype=pl.UInt8)
+    s1.dtype, s2.dtype
+    return s1, s2
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        ## Dataframe
+
+        A dataframe is a 2-dimensional data structure that contains uniquely named series and can hold multiple data types. Dataframes are more commonly used for data manipulation using the functionality of Polars.
+
+        The snippet below shows how to create a dataframe from a dictionary of lists:
+        """
+    )
+    return
+
+
+@app.cell
+def _(pl):
+    data = pl.DataFrame(
+        {
+            "Product ID": ["L51172", "M22586", "L51639", "L50250", "M20109"],
+            "Type": ["L", "M", "L", "L", "M"],
+            "Air temperature": [302.3, 300.8, 302.6, 300, 303.4],  # (K)
+            "Machine Failure": [False, True, False, False, True]
+        }
+    )
+    data
+    return (data,)
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        ### Inspecting a dataframe
+
+        Polars has various functions to explore the data in a dataframe. We will use the dataframe `data` defined above in our examples. Alongside we can also see a view of the dataframe rendered by `marimo` as the cells are executed.
+
+        ///note
+        We can also use `marimo`'s built in data-inspection elements/features such as  [`mo.ui.dataframe`](https://docs.marimo.io/api/inputs/dataframe/#marimo.ui.dataframe) & [`mo.ui.data_explorer`](https://docs.marimo.io/api/inputs/data_explorer/). For more check out our Polars tutorials at [`marimo learn`](https://marimo-team.github.io/learn/)!
+        """
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        """
+        #### Head
+
+        The function `head` shows the first rows of a dataframe. Unless specified, it shows the first 5 rows.
+        """
+    )
+    return
+
+
+@app.cell
+def _(data):
+    data.head(3)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        #### Glimpse
+
+        The function `glimpse` is an alternative to `head` to view the first few columns, but displays each line of the output corresponding to a single column. That way, it makes inspecting wider dataframes easier.
+        """
+    )
+    return
+
+
+@app.cell
+def _(data):
+    print(data.glimpse(return_as_string=True))
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        #### Tail
+
+        The `tail` function, just like its name suggests, shows the last rows of a dataframe. Unless the number of rows is specified, it will show the last 5 rows.
+        """
+    )
+    return
+
+
+@app.cell
+def _(data):
+    data.tail(3)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        #### Sample
+
+        `sample` can be used to show a specified number of randomly selected rows from the dataframe. Unless the number of rows is specified, it will show a single row. `sample` does not preserve order of the rows.
+        """
+    )
+    return
+
+
+@app.cell
+def _(data):
+    import random
+
+    random.seed(42)  # For reproducibility.
+
+    data.sample(3)
+    return (random,)
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        #### Describe
+
+        The function `describe` describes the summary statistics for all columns of a dataframe.
+        """
+    )
+    return
+
+
+@app.cell
+def _(data):
+    data.describe()
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        ## Schema
+
+        A schema is a mapping showing the datatype corresponding to every column of a dataframe. The schema of a dataframe can be viewed using the attribute `schema`.
+        """
+    )
+    return
+
+
+@app.cell
+def _(data):
+    data.schema
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""Since a schema is a mapping, it can be specified in the form of a Python dictionary. Then this dictionary can be used to specify the schema of a dataframe on definition. If not specified or the entry is `None`, Polars infers the datatype from the contents of the column. Note that if the schema is not specified, it will be inferred automatically by default.""")
+    return
+
+
+@app.cell
+def _(pl):
+    pl.DataFrame(
+        {
+            "Product ID": ["L51172", "M22586", "L51639", "L50250", "M20109"],
+            "Type": ["L", "M", "L", "L", "M"],
+            "Air temperature": [302.3, 300.8, 302.6, 300, 303.4],  # (K)
+            "Machine Failure": [False, True, False, False, True]
+        },
+        schema={"Product ID": pl.String, "Type": pl.String, "Air temperature": None, "Machine Failure": None},
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""Sometimes the automatically inferred schema is enough for some columns, but we might wish to override the inference of only some columns. We can specify the schema for those columns using `schema_overrides`.""")
+    return
+
+
+@app.cell
+def _(pl):
+    pl.DataFrame(
+        {
+            "Product ID": ["L51172", "M22586", "L51639", "L50250", "M20109"],
+            "Type": ["L", "M", "L", "L", "M"],
+            "Air temperature": [302.3, 300.8, 302.6, 300, 303.4],  # (K)
+            "Machine Failure": [False, True, False, False, True]
+        },
+        schema_overrides={"Air temperature": pl.Float32},
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        ### References
+
+        1. Polars documentation ([link](https://docs.pola.rs/api/python/stable/reference/datatypes.html))
+        """
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _():
+    import marimo as mo
+    import polars as pl
+    return mo, pl
+
+
+if __name__ == "__main__":
+    app.run()

probability/20_naive_bayes.py

@@ -0,0 +1,833 @@
+# /// script
+# requires-python = ">=3.10"
+# dependencies = [
+#     "marimo",
+#     "matplotlib==3.10.1",
+#     "scipy==1.15.2",
+#     "numpy==2.2.4",
+#     "polars==1.26.0",
+#     "plotly==5.18.0",
+#     "scikit-learn==1.6.1",
+# ]
+# ///
+
+import marimo
+
+__generated_with = "0.12.0"
+app = marimo.App(width="medium", app_title="Naive Bayes Classification")
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        # Naive Bayes Classification
+
+        _This notebook is a computational companion to ["Probability for Computer Scientists"](https://chrispiech.github.io/probabilityForComputerScientists/en/part5/naive_bayes/), by Stanford professor Chris Piech._
+
+        Naive Bayes is one of those classic machine learning algorithms that seems almost too simple to work, yet it's surprisingly effective for many classification tasks. I've always found it fascinating how this algorithm applies Bayes' theorem with a strong (but knowingly incorrect) "naive" assumption that all features are independent of each other.
+
+        In this notebook, we'll dive into why this supposedly "wrong" assumption still leads to good results. We'll walk through the training process, learn how to make predictions, and see some interactive visualizations that helped me understand the concept better when I was first learning it. We'll also explore why Naive Bayes excels particularly in text classification problems like spam filtering.
+
+        If you're new to Naive Bayes, I highly recommend checking out [this excellent explanation by Mahesh Huddar](https://youtu.be/XzSlEA4ck2I?si=AASeh_KP68BAbzy5), which provides a step-by-step walkthrough with a helpful example (which we take a dive into, down below).
+        """
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        ## Why "Naive"?
+
+        So why is it called "naive"? It's because the algorithm makes an assumption — it assumes all features are completely independent of each other when given the class label.
+
+        The math way of saying this is:
+
+        $$P(X_1, X_2, \ldots, X_n | Y) = P(X_1 | Y) \times P(X_2 | Y) \times \ldots \times P(X_n | Y) = \prod_{i=1}^{n} P(X_i | Y)$$
+
+        This independence assumption is almost always wrong in real data. Think about text classification — if you see the word "cloudy" in a weather report, you're much more likely to also see "rain" than you would be to see "sunshine". These words clearly depend on each other! Or in medical diagnosis, symptoms often occur together as part of syndromes.
+
+        But here's the cool part — even though we know this assumption is _technically_ wrong, the algorithm still works remarkably well in practice. By making this simplifying assumption, we:
+
+        - Make the math way easier to compute
+        - Need way less training data to get decent results 
+        - Can handle thousands of features without blowing up computationally
+        """
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        ## The Math Behind Naive Bayes
+
+        At its core, Naive Bayes is just an application of Bayes' theorem from our earlier probability notebooks. Let's break it down:
+
+        We have some features $\mathbf{X} = [X_1, X_2, \ldots, X_m]$ (like words in an email or symptoms of a disease) and we want to predict a class label $Y$ (like "spam/not spam" or "has disease/doesn't have disease").
+
+        What we're really trying to find is:
+
+        $$P(Y|\mathbf{X})$$
+
+        In other words, "what's the probability of a certain class given the features we observed?" Once we have these probabilities, we simply pick the class with the highest probability:
+
+        $$\hat{y} = \underset{y}{\operatorname{argmax}} \text{ } P(Y=y|\mathbf{X}=\mathbf{x})$$
+
+        Applying Bayes' theorem (from our earlier probability work), we get:
+
+        $$P(Y=y|\mathbf{X}=\mathbf{x}) = \frac{P(Y=y) \times P(\mathbf{X}=\mathbf{x}|Y=y)}{P(\mathbf{X}=\mathbf{x})}$$
+
+        Since we're comparing different possible classes for the same input features, the denominator $P(\mathbf{X}=\mathbf{x})$ is the same for all classes. So we can drop it and just compare:
+
+        $$\hat{y} = \underset{y}{\operatorname{argmax}} \text{ } P(Y=y) \times P(\mathbf{X}=\mathbf{x}|Y=y)$$
+
+        Here's where the "naive" part comes in. Calculating $P(\mathbf{X}=\mathbf{x}|Y=y)$ directly would be a computational nightmare - we'd need counts for every possible combination of feature values. Instead, we make that simplifying "naive" assumption that features are independent of each other:
+
+        $$P(\mathbf{X}=\mathbf{x}|Y=y) = \prod_{i=1}^{m} P(X_i=x_i|Y=y)$$
+
+        Which gives us our final formula:
+
+        $$\hat{y} = \underset{y}{\operatorname{argmax}} \text{ } P(Y=y) \times \prod_{i=1}^{m} P(X_i=x_i|Y=y)$$
+
+        In actual implementations, we usually use logarithms to avoid the numerical problems that come with multiplying many small probabilities (they can _underflow_ to zero):
+
+        $$\hat{y} = \underset{y}{\operatorname{argmax}} \text{ } \log P(Y=y) + \sum_{i=1}^{m} \log P(X_i=x_i|Y=y)$$
+
+        That's it! The really cool thing is that despite this massive simplification, the algorithm often gives surprisingly good results.
+        """
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""## Example Problem""")
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""Let's apply Naive Bayes principles to this data (Tennis Training Dataset):""")
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        ## A Simple Example: Play Tennis
+
+        Let's understand Naive Bayes with a classic example: predicting whether someone will play tennis based on weather conditions. This is the same example used in Mahesh Huddar's excellent video.
+
+        Our dataset has these features:
+        - **Outlook**: Sunny, Overcast, Rainy
+        - **Temperature**: Hot, Mild, Cool
+        - **Humidity**: High, Normal
+        - **Wind**: Strong, Weak
+
+        And the target variable:
+        - **Play Tennis**: Yes, No
+
+        ### Example Dataset
+        """
+    )
+
+    # Create a dataset matching the image (in dict format for proper table rendering)
+    example_data = [
+        {"Day": "D1", "Outlook": "Sunny", "Temperature": "Hot", "Humidity": "High", "Wind": "Weak", "Play Tennis": "No"},
+        {"Day": "D2", "Outlook": "Sunny", "Temperature": "Hot", "Humidity": "High", "Wind": "Strong", "Play Tennis": "No"},
+        {"Day": "D3", "Outlook": "Overcast", "Temperature": "Hot", "Humidity": "High", "Wind": "Weak", "Play Tennis": "Yes"},
+        {"Day": "D4", "Outlook": "Rain", "Temperature": "Mild", "Humidity": "High", "Wind": "Weak", "Play Tennis": "Yes"},
+        {"Day": "D5", "Outlook": "Rain", "Temperature": "Cool", "Humidity": "Normal", "Wind": "Weak", "Play Tennis": "Yes"},
+        {"Day": "D6", "Outlook": "Rain", "Temperature": "Cool", "Humidity": "Normal", "Wind": "Strong", "Play Tennis": "No"},
+        {"Day": "D7", "Outlook": "Overcast", "Temperature": "Cool", "Humidity": "Normal", "Wind": "Strong", "Play Tennis": "Yes"},
+        {"Day": "D8", "Outlook": "Sunny", "Temperature": "Mild", "Humidity": "High", "Wind": "Weak", "Play Tennis": "No"},
+        {"Day": "D9", "Outlook": "Sunny", "Temperature": "Cool", "Humidity": "Normal", "Wind": "Weak", "Play Tennis": "Yes"},
+        {"Day": "D10", "Outlook": "Rain", "Temperature": "Mild", "Humidity": "Normal", "Wind": "Weak", "Play Tennis": "Yes"},
+        {"Day": "D11", "Outlook": "Sunny", "Temperature": "Mild", "Humidity": "Normal", "Wind": "Strong", "Play Tennis": "Yes"},
+        {"Day": "D12", "Outlook": "Overcast", "Temperature": "Mild", "Humidity": "High", "Wind": "Strong", "Play Tennis": "Yes"},
+        {"Day": "D13", "Outlook": "Overcast", "Temperature": "Hot", "Humidity": "Normal", "Wind": "Weak", "Play Tennis": "Yes"},
+        {"Day": "D14", "Outlook": "Rain", "Temperature": "Mild", "Humidity": "High", "Wind": "Strong", "Play Tennis": "No"}
+    ]
+
+    # Display the tennis dataset using a table
+    example_table = mo.ui.table(
+        data=example_data,
+        selection=None
+    )
+
+    mo.vstack([
+        mo.md("#### Tennis Training Dataset"),
+        example_table
+    ])
+    return example_data, example_table
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        Let's predict whether someone will play tennis given these weather conditions:
+
+        - Outlook: Sunny
+        - Temperature: Cool
+        - Humidity: High
+        - Wind: Strong
+
+        Let's walk through the calculations step by step:
+
+        #### Step 1: Calculate Prior Probabilities
+
+        First, we calculate $P(Y=\text{Yes})$ and $P(Y=\text{No})$:
+
+        - $P(Y=\text{Yes}) = \frac{9}{14} = 0.64$
+        - $P(Y=\text{No}) = \frac{5}{14} = 0.36$
+
+        #### Step 2: Calculate Conditional Probabilities
+
+        Next, we calculate the conditional probabilities for each feature value given each class:
+        """
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(humidity_data, mo, outlook_data, summary_table, temp_data, wind_data):
+    # Display tables with appropriate styling
+    mo.vstack([
+        mo.md("#### Class Distribution"),
+        summary_table,
+        mo.md("#### Conditional Probabilities"),
+        mo.hstack([
+            mo.vstack([
+                mo.md("**Outlook**"),
+                mo.ui.table(
+                    data=outlook_data,
+                    selection=None
+                )
+            ]),
+            mo.vstack([
+                mo.md("**Temperature**"),
+                mo.ui.table(
+                    data=temp_data,
+                    selection=None
+                )
+            ])
+        ]),
+        mo.hstack([
+            mo.vstack([
+                mo.md("**Humidity**"),
+                mo.ui.table(
+                    data=humidity_data,
+                    selection=None
+                )
+            ]),
+            mo.vstack([
+                mo.md("**Wind**"),
+                mo.ui.table(
+                    data=wind_data,
+                    selection=None
+                )
+            ])
+        ])
+    ])
+    return
+
+
+@app.cell
+def _():
+    # DIY
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo, solution_accordion):
+    # Display the accordion
+    mo.accordion(solution_accordion)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        ### Try a Different Example
+
+        What if the conditions were different? Let's say:
+
+        - Outlook: Overcast
+        - Temperature: Hot
+        - Humidity: Normal
+        - Wind: Weak
+
+        Try working through this example on your own. If you get stuck, you can use the tables above and apply the same method we used in the solution.
+        """
+    )
+    return
+
+
+@app.cell
+def _():
+    # DIY
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        ## Interactive Naive Bayes
+
+        Let's explore Naive Bayes with an interactive visualization. This will help build intuition about how the algorithm makes predictions and how the naive independence assumption affects results.
+        """
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def gaussian_viz(
+    Ellipse,
+    GaussianNB,
+    ListedColormap,
+    class_sep_slider,
+    controls,
+    make_classification,
+    mo,
+    n_samples_slider,
+    noise_slider,
+    np,
+    pl,
+    plt,
+    regenerate_button,
+    train_test_split,
+):
+    # get values from sliders
+    class_sep = class_sep_slider.value
+    noise_val = noise_slider.value
+    n_samples = int(n_samples_slider.value)
+
+    # check if regenerate button was clicked
+    regenerate_state = regenerate_button.value
+
+    # make a dataset with current settings
+    X, y = make_classification(
+        n_samples=n_samples,
+        n_features=2,
+        n_redundant=0,
+        n_informative=2,
+        n_clusters_per_class=1,
+        class_sep=class_sep * (1 - noise_val),  # use noise to reduce separation
+        random_state=42 if not regenerate_state else np.random.randint(1000)
+    )
+
+    # put data in a dataframe
+    viz_df = pl.DataFrame({
+        "Feature1": X[:, 0],
+        "Feature2": X[:, 1],
+        "Class": y
+    })
+
+    # split into train/test
+    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)
+
+    # create naive bayes classifier
+    gnb = GaussianNB()
+    gnb.fit(X_train, y_train)
+
+    # setup grid for boundary visualization
+    h = 0.1  # step size
+    x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1
+    y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1
+    xx, yy = np.meshgrid(np.arange(x_min, x_max, h), np.arange(y_min, y_max, h))
+
+    # predict on grid points
+    grid_points = np.c_[xx.ravel(), yy.ravel()]
+    Z = gnb.predict(grid_points).reshape(xx.shape)
+
+    # calculate class stats
+    class0_mean = np.mean(X_train[y_train == 0], axis=0)
+    class1_mean = np.mean(X_train[y_train == 1], axis=0)
+    class0_var = np.var(X_train[y_train == 0], axis=0)
+    class1_var = np.var(X_train[y_train == 1], axis=0)
+
+    # format for display
+    class_stats = [
+        {"Class": "Class 0", "Feature1_Mean": f"{class0_mean[0]:.4f}", "Feature1_Variance": f"{class0_var[0]:.4f}",
+         "Feature2_Mean": f"{class0_mean[1]:.4f}", "Feature2_Variance": f"{class0_var[1]:.4f}"},
+        {"Class": "Class 1", "Feature1_Mean": f"{class1_mean[0]:.4f}", "Feature1_Variance": f"{class1_var[0]:.4f}",
+         "Feature2_Mean": f"{class1_mean[1]:.4f}", "Feature2_Variance": f"{class1_var[1]:.4f}"}
+    ]
+
+    # setup plot with two panels
+    fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(14, 6))
+
+    # colors for our plots
+    cmap_light = ListedColormap(['#FFAAAA', '#AAAAFF'])  # bg colors
+    cmap_bold = ListedColormap(['#FF0000', '#0000FF'])   # point colors
+
+    # left: decision boundary
+    ax1.contourf(xx, yy, Z, alpha=0.3, cmap=cmap_light)
+    scatter1 = ax1.scatter(X_train[:, 0], X_train[:, 1], c=y_train, 
+                          cmap=cmap_bold, edgecolor='k', s=50, alpha=0.8)
+    scatter2 = ax1.scatter(X_test[:, 0], X_test[:, 1], c=y_test, 
+                          cmap=cmap_bold, edgecolor='k', s=25, alpha=0.5)
+
+    ax1.set_xlabel('Feature 1')
+    ax1.set_ylabel('Feature 2')
+    ax1.set_title('Gaussian Naive Bayes Decision Boundary')
+    ax1.legend([scatter1.legend_elements()[0][0], scatter2.legend_elements()[0][0]],
+              ['Training Data', 'Test Data'], loc='upper right')
+
+    # right: distribution visualization
+    class0_data = viz_df.filter(pl.col("Class") == 0)
+    class1_data = viz_df.filter(pl.col("Class") == 1)
+
+    ax2.scatter(class0_data["Feature1"], class0_data["Feature2"], 
+               color='red', edgecolor='k', s=50, alpha=0.8, label='Class 0')
+    ax2.scatter(class1_data["Feature1"], class1_data["Feature2"], 
+               color='blue', edgecolor='k', s=50, alpha=0.8, label='Class 1')
+
+    # draw ellipses function
+    def plot_ellipse(ax, mean, cov, color):
+        vals, vecs = np.linalg.eigh(cov)
+        order = vals.argsort()[::-1]
+        vals = vals[order]
+        vecs = vecs[:, order]
+
+        theta = np.degrees(np.arctan2(*vecs[:, 0][::-1]))
+        width, height = 2 * np.sqrt(5.991 * vals)
+
+        ellip = Ellipse(xy=mean, width=width, height=height, angle=theta, 
+                       edgecolor=color, fc='None', lw=2, alpha=0.7)
+        ax.add_patch(ellip)
+
+    # add ellipses for each class accordingly
+    class0_cov = np.diag(np.var(X_train[y_train == 0], axis=0))
+    class1_cov = np.diag(np.var(X_train[y_train == 1], axis=0))
+
+    plot_ellipse(ax2, class0_mean, class0_cov, 'red')
+    plot_ellipse(ax2, class1_mean, class1_cov, 'blue')
+
+    ax2.set_xlabel('Feature 1')
+    ax2.set_ylabel('Feature 2')
+    ax2.set_title('Class-Conditional Distributions (Gaussian)')
+    ax2.legend(loc='upper right')
+
+    plt.tight_layout()
+    plt.gca()
+
+    # show interactive plot
+    mpl_fig = mo.mpl.interactive(fig)
+
+    # show parameters info
+    mo.md(
+        r"""
+        ### gaussian parameters by class
+
+        each feature follows a normal distribution per class. here are the parameters:
+        """
+    )
+
+    # make stats table
+    stats_table = mo.ui.table(
+        data=class_stats,
+        selection="single"
+    )
+
+    mo.md(
+        r"""
+        ### how it works
+
+        1. calculate mean & variance for each feature per class
+        2. use gaussian pdf to get probabilities for new points
+        3. apply bayes' theorem to pick most likely class
+
+        ellipses show the distributions, decision boundary is where probabilities equal
+        """
+    )
+
+    # stack everything together
+    mo.vstack([
+        controls.center(),
+        mpl_fig,
+        stats_table
+    ])
+    return (
+        X,
+        X_test,
+        X_train,
+        Z,
+        ax1,
+        ax2,
+        class0_cov,
+        class0_data,
+        class0_mean,
+        class0_var,
+        class1_cov,
+        class1_data,
+        class1_mean,
+        class1_var,
+        class_sep,
+        class_stats,
+        cmap_bold,
+        cmap_light,
+        fig,
+        gnb,
+        grid_points,
+        h,
+        mpl_fig,
+        n_samples,
+        noise_val,
+        plot_ellipse,
+        regenerate_state,
+        scatter1,
+        scatter2,
+        stats_table,
+        viz_df,
+        x_max,
+        x_min,
+        xx,
+        y,
+        y_max,
+        y_min,
+        y_test,
+        y_train,
+        yy,
+    )
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        ### what's going on in this demo?
+
+        Playing with the sliders changes how our data looks and how the classifier behaves. Class separation controls how far apart the two classes are — higher values make them easier to tell apart. The noise slider adds randomness by reducing that separation, making boundaries fuzzier and classification harder. More samples just gives you more data points to work with.
+
+        The left graph shows the decision boundary — that curved line where the classifier switches from predicting one class to another. Red and blue regions show where naive bayes would classify new points. The right graph shows the actual distribution of both classes, with those ellipses representing the gaussian distributions naive bayes is using internally.
+
+        Try cranking up the noise and watch how the boundary gets messier. increase separation and see how confident the classifier becomes. This is basically what's happening inside naive bayes — it's looking at each feature's distribution per class and making the best guess based on probabilities. The table below shows the actual parameters (means and variances) the model calculates.
+        """
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        ## Types of Naive Bayes Classifiers
+
+        ### Multinomial Naive Bayes
+        Ideal for text classification where features represent word counts or frequencies.
+
+        Mathematical form:
+
+        \[P(x_i|y) = \frac{\text{count}(x_i, y) + \alpha}{\sum_{i=1}^{|V|} \text{count}(x_i, y) + \alpha|V|}\]
+
+        where:
+
+        - \(\alpha\) is the smoothing parameter
+        - \(|V|\) is the size of the vocabulary
+        - \(\text{count}(x_i, y)\) is the count of feature \(i\) in class \(y\)
+
+        ### Bernoulli Naive Bayes
+        Best for binary features (0/1) — either a word appears or it doesn't.
+
+        Mathematical form:
+
+        \[P(x_i|y) = p_{iy}^{x_i}(1-p_{iy})^{(1-x_i)}\]
+
+        where:
+
+        - \(p_{iy}\) is the probability of feature \(i\) occurring in class \(y\)
+        - \(x_i\) is 1 if the feature is present, 0 otherwise
+
+        ### Gaussian Naive Bayes
+        Designed for continuous features, assuming they follow a normal distribution.
+
+        Mathematical form:
+
+        \[P(x_i|y) = \frac{1}{\sqrt{2\pi\sigma_y^2}} \exp\left(-\frac{(x_i - \mu_y)^2}{2\sigma_y^2}\right)\]
+
+        where:
+
+        - \(\mu_y\) is the mean of feature values for class \(y\)
+        - \(\sigma_y^2\) is the variance of feature values for class \(y\)
+
+        ### Complement Naive Bayes
+        Particularly effective for imbalanced datasets.
+
+        Mathematical form:
+
+        \[P(x_i|y) = \frac{\text{count}(x_i, \bar{y}) + \alpha}{\sum_{i=1}^{|V|} \text{count}(x_i, \bar{y}) + \alpha|V|}\]
+
+        where:
+
+        - \(\bar{y}\) represents all classes except \(y\)
+        - Other parameters are similar to Multinomial Naive Bayes
+        """
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        ## 🤔 Test Your Understanding
+
+        Test your understanding of Naive Bayes with these statements:
+
+        /// details | Multiplying small probabilities in Naive Bayes can lead to numerical underflow.
+        ✅ **Correct!** Multiplying many small probabilities can indeed lead to numerical underflow.
+
+        That's why in practice, we often use log probabilities and add them instead of multiplying the original probabilities. This prevents numerical underflow and improves computational stability.
+        ///
+
+        /// details | Laplace smoothing is unnecessary if your training data covers all possible feature values.
+        ❌ **Incorrect.** Laplace smoothing is still beneficial even with complete feature coverage.
+
+        While Laplace smoothing is crucial for handling unseen feature values, it also helps with small sample sizes by preventing overfitting to the training data. Even with complete feature coverage, some combinations might have very few examples, leading to unreliable probability estimates.
+        ///
+
+        /// details | Naive Bayes performs poorly on high-dimensional data compared to other classifiers.
+        ❌ **Incorrect.** Naive Bayes actually excels with high-dimensional data.
+
+        Due to its simplicity and the independence assumption, Naive Bayes scales very well to high-dimensional data. It's particularly effective for text classification where each word is a dimension and there can be thousands of dimensions. Other classifiers might overfit in such high-dimensional spaces.
+        ///
+
+        /// details | For text classification, Multinomial Naive Bayes typically outperforms Gaussian Naive Bayes.
+        ✅ **Correct!** Multinomial NB is better suited for text classification than Gaussian NB.
+
+        Text data typically involves discrete counts (word frequencies) which align better with a multinomial distribution. Gaussian Naive Bayes assumes features follow a normal distribution, which doesn't match the distribution of word frequencies in text documents.
+        ///
+        """
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        ## Summary
+
+        Throughout this notebook, we've explored Naive Bayes classification. What makes this algorithm particularly interesting is its elegant simplicity combined with surprising effectiveness. Despite making what seems like an overly simplistic assumption — that features are independent given the class — it consistently delivers reasonable performance across a wide range of applications.
+
+        The algorithm's power lies in its probabilistic foundation, built upon Bayes' theorem. During training, it simply learns probability distributions: the likelihood of seeing each class (prior probabilities) and the probability of feature values within each class (conditional probabilities). When making predictions, it combines these probabilities using the naive independence assumption, which dramatically simplifies the computation while still maintaining remarkable predictive power.
+
+        We've seen how different variants of Naive Bayes adapt to various types of data. Multinomial Naive Bayes excels at text classification by modeling word frequencies, Bernoulli Naive Bayes handles binary features elegantly, and Gaussian Naive Bayes tackles continuous data through normal distributions. Each variant maintains the core simplicity of the algorithm while adapting its probability calculations to match the data's characteristics.
+
+        Perhaps most importantly, we've learned that sometimes the most straightforward approaches can be the most practical. Naive Bayes demonstrates that a simple model, well-understood and properly applied, can often outperform more complex alternatives, especially in domains like text classification or when working with limited computational resources or training data.
+        """
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""appendix (helper code)""")
+    return
+
+
+@app.cell
+def _():
+    import marimo as mo
+    return (mo,)
+
+
+@app.cell
+def init_imports():
+    # imports for our notebook
+    import numpy as np
+    import matplotlib.pyplot as plt
+    import polars as pl
+    from scipy import stats
+    from sklearn.naive_bayes import GaussianNB
+    from sklearn.datasets import make_classification
+    from sklearn.model_selection import train_test_split
+    from matplotlib.colors import ListedColormap
+    from matplotlib.patches import Ellipse
+
+    # for consistent results
+    np.random.seed(42)
+
+    # nicer plots
+    plt.style.use('seaborn-v0_8-darkgrid')
+    return (
+        Ellipse,
+        GaussianNB,
+        ListedColormap,
+        make_classification,
+        np,
+        pl,
+        plt,
+        stats,
+        train_test_split,
+    )
+
+
+@app.cell(hide_code=True)
+def _(example_data, mo):
+    # occurrences count in example data
+    yes_count = sum(1 for row in example_data if row["Play Tennis"] == "Yes")
+    no_count = sum(1 for row in example_data if row["Play Tennis"] == "No")
+    total = len(example_data)
+
+    # summary table with dict format
+    summary_data = [
+        {"Class": "Yes", "Count": f"{yes_count}", "Probability": f"{yes_count/total:.2f}"},
+        {"Class": "No", "Count": f"{no_count}", "Probability": f"{no_count/total:.2f}"},
+        {"Class": "Total", "Count": f"{total}", "Probability": "1.00"}
+    ]
+
+    summary_table = mo.ui.table(
+        data=summary_data,
+        selection=None
+    )
+
+    # tables for conditional probabilities matching the image (in dict format)
+    outlook_data = [
+        {"Outlook": "Sunny", "Y": "2/9", "N": "3/5"},
+        {"Outlook": "Overcast", "Y": "4/9", "N": "0"},
+        {"Outlook": "Rain", "Y": "3/9", "N": "2/5"}
+    ]
+
+    temp_data = [
+        {"Temperature": "Hot", "Y": "2/9", "N": "2/5"},
+        {"Temperature": "Mild", "Y": "4/9", "N": "2/5"},
+        {"Temperature": "Cool", "Y": "3/9", "N": "1/5"}
+    ]
+
+    humidity_data = [
+        {"Humidity": "High", "Y": "3/9", "N": "4/5"},
+        {"Humidity": "Normal", "Y": "6/9", "N": "1/5"}
+    ]
+
+    wind_data = [
+        {"Wind": "Strong", "Y": "3/9", "N": "3/5"},
+        {"Wind": "Weak", "Y": "6/9", "N": "2/5"}
+    ]
+    return (
+        humidity_data,
+        no_count,
+        outlook_data,
+        summary_data,
+        summary_table,
+        temp_data,
+        total,
+        wind_data,
+        yes_count,
+    )
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    # accordion with solution (step-by-step)
+    solution_accordion = {
+        "step-by-step solution (click to expand)": mo.md(r"""
+        #### step 1: gather probabilities
+
+        from our tables:
+
+        **prior probabilities:**
+
+        - $P(Yes) = 9/14 = 0.64$
+        - $P(No) = 5/14 = 0.36$
+
+        **conditional probabilities:**
+
+        - $P(Outlook=Sunny|Yes) = 2/9$
+        - $P(Outlook=Sunny|No) = 3/5$
+        - $P(Temperature=Cool|Yes) = 3/9$
+        - $P(Temperature=Cool|No) = 1/5$
+        - $P(Humidity=High|Yes) = 3/9$
+        - $P(Humidity=High|No) = 4/5$
+        - $P(Wind=Strong|Yes) = 3/9$
+        - $P(Wind=Strong|No) = 3/5$
+
+        #### step 2: calculate for yes
+
+        $P(Yes) \times P(Sunny|Yes) \times P(Cool|Yes) \times P(High|Yes) \times P(Strong|Yes)$
+
+        $= \frac{9}{14} \times \frac{2}{9} \times \frac{3}{9} \times \frac{3}{9} \times \frac{3}{9}$
+
+        $= \frac{9}{14} \times \frac{2 \times 3 \times 3 \times 3}{9^4}$
+
+        $= \frac{9}{14} \times \frac{54}{6561}$
+
+        $= \frac{9 \times 54}{14 \times 6561}$
+
+        $= \frac{486}{91854}$
+
+        $= 0.0053$
+
+        #### step 3: calculate for no
+
+        $P(No) \times P(Sunny|No) \times P(Cool|No) \times P(High|No) \times P(Strong|No)$
+
+        $= \frac{5}{14} \times \frac{3}{5} \times \frac{1}{5} \times \frac{4}{5} \times \frac{3}{5}$
+
+        $= \frac{5}{14} \times \frac{3 \times 1 \times 4 \times 3}{5^4}$
+
+        $= \frac{5}{14} \times \frac{36}{625}$
+
+        $= \frac{5 \times 36}{14 \times 625}$
+
+        $= \frac{180}{8750}$
+
+        $= 0.0206$
+
+        #### step 4: normalize
+
+        sum of probabilities: $0.0053 + 0.0206 = 0.0259$
+
+        normalizing:
+
+        - $P(Yes|evidence) = \frac{0.0053}{0.0259} = 0.205$ (20.5%)
+        - $P(No|evidence) = \frac{0.0206}{0.0259} = 0.795$ (79.5%)
+
+        #### step 5: predict
+
+        since $P(No|evidence) > P(Yes|evidence)$, prediction: **No**
+
+        person would **not play tennis** under these conditions.
+        """)
+    }
+    return (solution_accordion,)
+
+
+@app.cell(hide_code=True)
+def create_gaussian_controls(mo):
+    # sliders for controlling viz parameters
+    class_sep_slider = mo.ui.slider(1.0, 3.0, value=1.5, label="Class Separation")
+    noise_slider = mo.ui.slider(0.1, 0.5, step=0.1, value=0.1, label="Noise (reduces class separation)")
+    n_samples_slider = mo.ui.slider(50, 200, value=100, step=10, label="Number of Samples")
+
+    # Create a run button to regenerate data
+    regenerate_button = mo.ui.run_button(label="Regenerate Data", kind="success")
+
+    # stack controls vertically
+    controls = mo.vstack([
+        mo.md("### visualization controls"),
+        class_sep_slider,
+        noise_slider,
+        n_samples_slider,
+        regenerate_button
+    ])
+    return (
+        class_sep_slider,
+        controls,
+        n_samples_slider,
+        noise_slider,
+        regenerate_button,
+    )
+
+
+if __name__ == "__main__":
+    app.run()

probability/21_logistic_regression.py

@@ -0,0 +1,530 @@
+# /// script
+# requires-python = ">=3.10"
+# dependencies = [
+#     "marimo",
+#     "matplotlib==3.10.1",
+#     "numpy==2.2.4",
+#     "drawdata==0.3.7",
+#     "scikit-learn==1.6.1",
+#     "polars==1.26.0",
+# ]
+# ///
+
+import marimo
+
+__generated_with = "0.12.5"
+app = marimo.App(width="medium", app_title="Logistic Regression")
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        # Logistic Regression
+
+        _This notebook is a computational companion to ["Probability for Computer Scientists"](https://chrispiech.github.io/probabilityForComputerScientists/en/part5/log_regression/), by Stanford professor Chris Piech._
+
+        Logistic regression learns a function approximating $P(y|x)$, and can be used to make a classifier. It makes the central assumption that $P(y|x)$ can be approximated as a sigmoid function applied to a linear combination of input features. It is particularly important to learn because logistic regression is the basic building block of artificial neural networks.
+        """
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        ## The Binary Classification Problem
+
+        Imagine situations where we would like to know:
+
+        - The eligibility of getting a bank loan given the value of credit score ($x_{credit\_score}$) and monthly income ($x_{income}$)
+        - Identifying a tumor as benign or malignant given its size ($x_{tumor\_size}$)
+        - Classifying an email as promotional given the number of occurrences for some keywords like {'win', 'gift', 'discount'} ($x_{n\_win}$, $x_{n\_gift}$, $x_{n\_discount}$)
+        - Finding a monetary transaction as fraudulent given the time of occurrence ($x_{time\_stamp}$) and amount ($x_{amount}$)
+
+        These problems occur frequently in real life & can be dealt with machine learning. All such problems come under the umbrella of what is known as Classification. In each scenario, only one of the two possible outcomes can occur, hence these are specifically known as Binary Classification problems.
+
+        ### How Does A Machine Perform Classification?
+
+        During the inference, the goal is to have the ML model predict the class label for a given set of feature values.
+
+        Specifically, a binary classification model estimates two probabilities $p_0$ & $p_1$ for 'class-0' and 'class-1' respectively where $p_0 + p_1 = 1$.
+
+        The predicted label depends on $\max(p_0, p_1)$ i.e., it's the one which is most probable based on the given features.
+
+        In logistic regression, $p_1$ (i.e., success probability) is compared with a predefined threshold $p$ to predict the class label like below:
+
+        $$\text{predicted class} = 
+        \begin{cases}
+        1, & \text{if } p_1 \geq p \\
+        0, & \text{otherwise}
+        \end{cases}$$
+
+        To keep the notation simple and consistent, we will denote the success probability as $p$, and failure probability as $(1-p)$ instead of $p_1$ and $p_0$ respectively.
+        """
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        ## Why NOT Linear Regression?
+
+        Can't we really use linear regression to address classification? The answer is NO! The key issue is that probabilities must be between 0 and 1 and linear regression can output any real number.
+
+        If we tried using linear regression directly:
+        $$p = \beta_0 + \beta_1 \cdot x_{feature}$$
+
+        This creates a problem: the right side can produce any value in $\mathbb{R}$ (all real numbers), but a probability $p$ must be confined to the range $(0,1)$.
+
+        Can we convert $(\beta_0 + \beta_1 \cdot x_{tumor\_size})$ to something belonging to $(0,1)$? That may work as an estimate of a probability! The answer is YES!
+
+        We need a converter (a function), say, $g()$ that will connect $p \in (0,1)$ to $(\beta_0 + \beta_1 \cdot x_{tumor\_size}) \in \mathbb{R}$.
+
+        The solution is to use a "link function" that maps from any real number to a valid probability range. This is where the sigmoid function comes in.
+        """
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo, np, plt):
+    # plot sigmoid to evidentiate above statements
+    _fig, ax = plt.subplots(figsize=(10, 6))
+
+    # x values
+    x = np.linspace(-10, 10, 1000)
+
+    # sigmoid formula
+    def sigmoid(z):
+        return 1 / (1 + np.exp(-z))
+
+    y = sigmoid(x)
+
+    # plot
+    ax.plot(x, y, 'b-', linewidth=2)
+
+    ax.axhline(y=0, color='k', linestyle='-', alpha=0.3)
+    ax.axhline(y=1, color='k', linestyle='-', alpha=0.3)
+    ax.axhline(y=0.5, color='r', linestyle='--', alpha=0.5)
+
+    # vertical line at x=0
+    ax.axvline(x=0, color='k', linestyle='-', alpha=0.3)
+
+    # annotations
+    ax.text(1, 0.85, r'$\sigma(z) = \frac{1}{1 + e^{-z}}$', fontsize=14)
+    ax.text(-9, 0.1, 'As z → -∞, σ(z) → 0', fontsize=12)
+    ax.text(3, 0.9, 'As z → ∞, σ(z) → 1', fontsize=12)
+    ax.text(0.5, 0.4, 'σ(0) = 0.5', fontsize=12)
+
+    # labels and title
+    ax.set_xlabel('z', fontsize=14)
+    ax.set_ylabel('σ(z)', fontsize=14)
+    ax.set_title('Sigmoid Function', fontsize=16)
+
+    # axis limits set
+    ax.set_xlim(-10, 10)
+    ax.set_ylim(-0.1, 1.1)
+
+    # grid
+    ax.grid(True, alpha=0.3)
+
+    mo.mpl.interactive(_fig)
+    return ax, sigmoid, x, y
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        **Figure**: The sigmoid function maps any real number to a value between 0 and 1, making it perfect for representing probabilities.
+
+        /// note
+        For more information about the sigmoid function, head over to [this detailed notebook](http://marimo.app/https://github.com/marimo-team/deepml-notebooks/blob/main/problems/problem-22/notebook.py) for more insights.
+        ///
+        """
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        ## The Core Concept (math)
+
+        Logistic regression models the probability of class 1 using the sigmoid function:
+
+        $$P(Y=1|X=x) = \sigma(z) \text{ where } z = \theta_0 + \sum_{i=1}^m \theta_i x_i$$
+
+        The sigmoid function $\sigma(z)$ transforms any real number into a probability between 0 and 1:
+
+        $$\sigma(z) = \frac{1}{1+ e^{-z}}$$
+
+        This can be written more compactly using vector notation:
+
+        $$P(Y=1|\mathbf{X}=\mathbf{x}) =\sigma(\mathbf{\theta}^T\mathbf{x}) \quad \text{ where we always set $x_0$ to be 1}$$
+
+        $$P(Y=0|\mathbf{X}=\mathbf{x}) =1-\sigma(\mathbf{\theta}^T\mathbf{x}) \quad \text{ by total law of probability}$$
+
+        Where $\theta$ represents the model parameters that need to be learned from data, and $x$ is the feature vector (with $x_0=1$ to account for the intercept term).
+
+        > **Note:** For the detailed mathematical derivation of how these parameters are learned through Maximum Likelihood Estimation (MLE) and Gradient Descent (GD), please refer to [Chris Piech's original material](https://chrispiech.github.io/probabilityForComputerScientists/en/part5/log_regression/). The mathematical details are elegant but beyond the scope of this notebook topic (which is confined to Logistic Regression).
+        """
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        ### Linear Decision Boundary
+
+        A key characteristic of logistic regression is that it creates a linear decision boundary. When the model predicts, it's effectively dividing the feature space with a straight line (in 2D) or hyperplane (in higher dimensions). It is actually a straight line (of the form $y = mx + c$).
+
+        Recall the prediction rule:
+        $$\text{predicted class} = 
+        \begin{cases}
+        1, & \text{if } p \geq \theta_0 + \theta_1 \cdot x_{tumor\_size} \Rightarrow \log\frac{p}{1-p} \\
+        0, & \text{otherwise}
+        \end{cases}$$
+
+        For a two-feature model, the decision boundary where $P(Y=1|X=x) = 0.5$ occurs at:
+        $$\theta_0 + \theta_1 x_1 + \theta_2 x_2 = 0$$
+
+        A simple logistic regression predicts the class label by identifying the regions on either side of a straight line (or hyperplane in general), hence it's a _linear_ classifier.
+
+        This linear nature makes logistic regression effective for linearly separable classes but limited when dealing with more complex patterns.
+        """
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md("""### Visual: Linear Separability and Classification""")
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo, np, plt):
+    # show relevant comparison to the above concepts/statements
+
+    fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(12, 5))
+
+    # Linear separable data
+    np.random.seed(42)
+    X1 = np.random.randn(100, 2) - 2
+    X2 = np.random.randn(100, 2) + 2
+
+    ax1.scatter(X1[:, 0], X1[:, 1], color='blue', alpha=0.5)
+    ax1.scatter(X2[:, 0], X2[:, 1], color='red', alpha=0.5)
+
+    # Decision boundary (line)
+    ax1.plot([-5, 5], [5, -5], 'k--', linewidth=2)
+    ax1.set_xlim(-5, 5)
+    ax1.set_ylim(-5, 5)
+    ax1.set_title('Linearly Separable Classes')
+
+    # non-linear separable data
+    radius = 2
+    theta = np.linspace(0, 2*np.pi, 100)
+
+    # Outer circle points (class 1)
+    outer_x = 3 * np.cos(theta)
+    outer_y = 3 * np.sin(theta)
+    # Inner circle points (class 2)
+    inner_x = 1.5 * np.cos(theta) + np.random.randn(100) * 0.2
+    inner_y = 1.5 * np.sin(theta) + np.random.randn(100) * 0.2
+
+    ax2.scatter(outer_x, outer_y, color='blue', alpha=0.5)
+    ax2.scatter(inner_x, inner_y, color='red', alpha=0.5)
+
+    # Attempt to draw a linear boundary (which won't work well) proving the point
+    ax2.plot([-5, 5], [2, 2], 'k--', linewidth=2)
+
+    ax2.set_xlim(-5, 5)
+    ax2.set_ylim(-5, 5)
+    ax2.set_title('Non-Linearly Separable Classes')
+
+    fig.tight_layout()
+    mo.mpl.interactive(fig)
+    return (
+        X1,
+        X2,
+        ax1,
+        ax2,
+        fig,
+        inner_x,
+        inner_y,
+        outer_x,
+        outer_y,
+        radius,
+        theta,
+    )
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""**Figure**: On the left, the classes are linearly separable as the boundary is a straight line. However, they are not linearly separable on the right, where no straight line can properly separate the two classes.""")
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        Logistic regression is typically trained using MLE - finding the parameters $\theta$ that make our observed data most probable.
+
+        The optimization process generally uses GD (or its variants) to iteratively improve the parameters. The gradient has a surprisingly elegant form:
+
+        $$\frac{\partial LL(\theta)}{\partial \theta_j} = \sum_{i=1}^n \left[
+        y^{(i)} - \sigma(\theta^T x^{(i)})
+        \right] x_j^{(i)}$$
+
+        This shows that the update to each parameter depends on the prediction error (actual - predicted) multiplied by the feature value.
+
+        For those interested in the complete mathematical derivation, including log likelihood calculation and the detailed steps of GD (and relevant pseudocode followed for training), please see the [original lecture notes](https://chrispiech.github.io/probabilityForComputerScientists/en/part5/log_regression/).
+        """
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(controls, mo, widget):
+    # create the layout
+    mo.vstack([
+        mo.md("## Interactive drawing demo\nDraw points of two different classes and see how logistic regression separates them. _The interactive demo was adapted and improvised from [Vincent Warmerdam's](https://github.com/koaning) code [here](https://github.com/probabl-ai/youtube-appendix/blob/main/04-drawing-data/notebook.ipynb)_."),
+        controls,
+        widget
+    ])
+    return
+
+
+@app.cell(hide_code=True)
+def _(LogisticRegression, mo, np, plt, run_button, widget):
+    warning_msg = mo.md(""" /// warning
+    Need more data, please draw points of at least two different colors in the scatter widget
+    """)
+
+    # mo.stop if button isn't clicked yet
+    mo.stop(
+        not run_button.value,
+        mo.md(""" /// tip 
+        click 'Run Logistic Regression' to see the model
+        """)
+    )
+
+    # get data from widget (can also use as_pandas)
+    df = widget.data_as_polars
+
+    # display appropriate warning
+    mo.stop(
+        df.is_empty() or df['color'].n_unique() < 2,
+        warning_msg
+    )
+
+    # extract features and labels
+    X = df[['x', 'y']].to_numpy()
+    y_colors = df['color'].to_numpy()
+
+    # fit logistic regression model
+    model = LogisticRegression()
+    model.fit(X, y_colors)
+
+    # create grid for the viz
+    x_min, x_max = X[:, 0].min() - 0.5, X[:, 0].max() + 0.5
+    y_min, y_max = X[:, 1].min() - 0.5, X[:, 1].max() + 0.5
+    xx, yy = np.meshgrid(
+        np.linspace(x_min, x_max, 100),
+        np.linspace(y_min, y_max, 100)
+    )
+
+    # get probability predictions
+    Z = model.predict_proba(np.c_[xx.ravel(), yy.ravel()])[:, 1]
+    Z = Z.reshape(xx.shape)
+
+    # create figure
+    _fig, ax_fig = plt.subplots(figsize=(12, 8))
+
+    # plot decision boundary (probability contours)
+    contour = ax_fig.contourf(
+        xx, yy, Z, 
+        levels=np.linspace(0, 1, 11),
+        alpha=0.7,
+        cmap="RdBu_r"
+    )
+
+    # plot decision boundary line (probability = 0.5)
+    ax_fig.contour(
+        xx, yy, Z,
+        levels=[0.5],
+        colors='k',
+        linewidths=2
+    )
+
+    # plot the data points (use same colors as in the widget)
+    ax_fig.scatter(X[:, 0], X[:, 1], c=y_colors, edgecolor='k', s=80)
+
+    # colorbar
+    plt.colorbar(contour, ax=ax_fig)
+
+    # labels and title
+    ax_fig.set_xlabel('x')
+    ax_fig.set_ylabel('y')
+    ax_fig.set_title('Logistic Regression')
+
+    # model params
+    coef = model.coef_[0]
+    intercept = model.intercept_[0]
+    equation = f"log(p/(1-p)) = {intercept:.2f} + {coef[0]:.3f}x₁ + {coef[1]:.3f}x₂"
+
+    # relevant info in regards to regression
+    model_info = mo.md(f"""
+    ### Logistic regression model
+
+    **Equation**: {equation}
+
+    **Decision boundary**: probability = 0.5
+
+    **Accuracy**: {model.score(X, y_colors):.2f}
+    """)
+
+    # show results vertically stacked
+    mo.vstack([
+        mo.mpl.interactive(_fig),
+        model_info
+    ])
+    return (
+        X,
+        Z,
+        ax_fig,
+        coef,
+        contour,
+        df,
+        equation,
+        intercept,
+        model,
+        model_info,
+        warning_msg,
+        x_max,
+        x_min,
+        xx,
+        y_colors,
+        y_max,
+        y_min,
+        yy,
+    )
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        ## 🤔 Key Takeaways
+
+        Click on the statements below that you think are correct to verify your understanding:
+
+        /// details | Logistic regression tries to find parameters (θ) that minimize the error between predicted and actual values using ordinary least squares.
+        ❌ **Incorrect.** Logistic regression uses maximum likelihood estimation (MLE), not ordinary least squares. It finds parameters that maximize the probability of observing the training data, which is different from minimizing squared errors as in linear regression.
+        ///
+
+        /// details | The sigmoid function maps any real number to a value between 0 and 1, which allows logistic regression to output probabilities.
+        ✅ **Correct!** The sigmoid function σ(z) = 1/(1+e^(-z)) takes any real number as input and outputs a value between 0 and 1. This is perfect for representing probabilities and is a key component of logistic regression.
+        ///
+
+        /// details | The decision boundary in logistic regression is always a straight line, regardless of the data's complexity.
+        ✅ **Correct!** Standard logistic regression produces a linear decision boundary (a straight line in 2D or a hyperplane in higher dimensions). This is why it works well for linearly separable data but struggles with more complex patterns, like concentric circles (as you might've noticed from the interactive demo).
+        ///
+
+        /// details | The logistic regression model params are typically initialized to random values and refined through gradient descent.
+        ✅ **Correct!** Parameters are often initialized to zeros or small random values, then updated iteratively using gradient descent (or ascent for maximizing likelihood) until convergence.
+        ///
+
+        /// details | Logistic regression can naturally handle multi-class classification problems without any modifications.
+        ❌ **Incorrect.** Standard logistic regression is inherently a binary classifier. To handle multi-class classification, techniques like one-vs-rest or softmax regression are typically used.
+        ///
+        """
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        ## Summary
+
+        So we've just explored logistic regression. Despite its name (seriously though, why not call it "logistic classification"?), it's actually quite elegant in how it transforms a simple linear model into a powerful decision _boundary_ maker.
+
+        The training process boils down to finding the values of θ that maximize the likelihood of seeing our training data. What's super cool is that even though the math looks _scary_ at first, the gradient has this surprisingly simple form: just the error (y - predicted) multiplied by the feature values.
+
+        Two key insights to remember:
+
+        - Logistic regression creates a _linear_ decision boundary, so it works great for linearly separable classes but struggles with more _complex_ patterns
+        - It directly gives you probabilities, not just classifications, which is incredibly useful when you need confidence measures
+        """
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(
+        r"""
+        Additional resources referred to:
+
+        - [Logistic Regression Tutorial by _Koushik Khan_](https://koushikkhan.github.io/resources/pdf/tutorials/logistic_regression_tutorial.pdf)
+        """
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""Appendix (helper code)""")
+    return
+
+
+@app.cell
+def _():
+    import marimo as mo
+    return (mo,)
+
+
+@app.cell
+def init_imports():
+    # imports for our notebook
+    import numpy as np
+    import matplotlib.pyplot as plt
+    from drawdata import ScatterWidget
+    from sklearn.linear_model import LogisticRegression
+
+
+    # for consistent results
+    np.random.seed(42)
+
+    # nicer plots
+    plt.style.use('seaborn-v0_8-darkgrid')
+    return LogisticRegression, ScatterWidget, np, plt
+
+
+@app.cell(hide_code=True)
+def _(ScatterWidget, mo):
+    # drawing widget
+    widget = mo.ui.anywidget(ScatterWidget())
+
+    # run_button to run model
+    run_button = mo.ui.run_button(label="Run Logistic Regression", kind="success")
+
+    # stack controls
+    controls = mo.hstack([run_button])
+    return controls, run_button, widget
+
+
+if __name__ == "__main__":
+    app.run()