> For the complete documentation index, see [llms.txt](https://docs.algenta.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.algenta.ai/machine-learning/ml-linear-models.md).

# Trainable linear models

Trainable logistic regression and elastic-net linear models with exact coordinate-descent math.

`linear_models` replaces `sklearn.linear_model` (`LogisticRegression`, `ElasticNet`, `Lasso`) and statsmodels `Logit` with pure Mojo, zero Python imports. It complements the closed-form OLS/ridge kernel (`regression.mojo`) with iterative fits: L2-regularized logistic regression via damped Newton (IRLS) with backtracking line search, and elastic-net linear regression via cyclic coordinate descent with exact soft-thresholding. Fully deterministic — there is no RNG anywhere in the kernel.

## The modules

| Module          | What it does                                                                                                                                                                                                                                                                                   |
| --------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `linear_models` | L2-regularized logistic regression via damped Newton (IRLS) with backtracking line search, and elastic-net linear regression via cyclic coordinate descent with exact soft-thresholding — replacing `sklearn.linear_model`'s `LogisticRegression`/`ElasticNet`/`Lasso` and statsmodels `Logit` |

## Functions

| Function                                            | What it does                                                                                                                                                                                                                                                                          |
| --------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `logistic_fit(X, y, l2, max_iter, tol)`             | L2-regularized logistic regression on flat row-major X (bias never penalized). Each Newton step solves `(X'WX + l2 I) d = grad` and halves the step if the penalized log-likelihood would decrease. Returns weights, bias, `converged`, `n_iter`, and the unpenalized log-likelihood. |
| `logistic_predict_proba(X, weights, bias)`          | `P(y=1) = sigmoid(X·w + bias)` per row, with a numerically stable sigmoid.                                                                                                                                                                                                            |
| `logistic_std_errors(X, y, weights, bias, l2)`      | Standard errors from the inverse observed information `(X'WX + l2 I)^-1`, ordered `[se_w0, .., se_w{p-1}, se_bias]`; pass `l2=0` for plain-MLE, statsmodels-style SEs.                                                                                                                |
| `elasticnet_fit(X, y, l1, l2, max_iter, tol=1e-10)` | Elastic net minimizing `(1/2n)·RSS + l1·L1(w) + (l2/2)·L2(w)^2` (bias unpenalized). `l1=l2=0` reproduces OLS; `l1=0` is ridge on this scaling.                                                                                                                                        |

## Guarantees and edge behavior

Soft-thresholding is exact, so irrelevant features land on 0.0 **exactly**, not near zero — the tests require an orthogonal irrelevant feature to be below 1e-15 and an all-zero column to be pinned at literal `0.0`. Malformed input (empty y, ragged X) returns a default result with `converged=False` rather than trapping. With `l2 = 0` on separable data, logistic weights grow until `max_iter` while probabilities stay finite and well-ordered; a small ridge (the tests use `l2 = 1e-3`) keeps the optimum finite and the fitted probabilities strictly increasing in x.

## Verification against closed-form references

The tests pin values that are checkable by hand (and against statsmodels):

* **Exact 2×2-table MLE**: groups with event rates 1/4 and 3/4 give the closed form `w = 2 ln 3 = 2.1972245773362196`, `bias = ln(1/3) = -1.0986122886681098`; `logistic_fit` matches both to 1e-6, with log-likelihood `-4.498681156950466` and fitted probabilities recovering the group rates 0.25 / 0.75.
* **Standard errors**: on the same table the hand-inverted information matrix gives `se_w = sqrt(8/3) = 1.632993161855452`, `se_bias = sqrt(4/3) = 1.1547005383792515`; matched to 1e-4.
* **Elastic net fixed points**: on a centered orthogonal design with `l1=0.5, l2=0.25`, `w0 = S(4, 0.5)/(2 + 0.25) = 1.5555555555555556` to 1e-12; lasso-only shrinkage `w0 = S(4, 0.5)/2 = 1.75` exactly; ridge-only `w = 4/3` on centered data; `l1=l2=0` recovers OLS (`w=2, b=1`); overwhelming `l1=100` kills all weights and leaves `bias = mean(y)`.

**Next:** promote what you trained with [statistical model comparison](/machine-learning/ml-model-comparison.md), or browse the [engine catalog](/engine-runtime/catalog.md).


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.algenta.ai/machine-learning/ml-linear-models.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.