profile pic
⌘ '
raccourcis clavier
Search

Logistic regression

  • Étiquette

    • publié à

    • modifié à

    • durée

      1 min de lecture

    • source

      llms.txt

    😄 fun fact: actually better for classification instead of regression problems

    Assume there is a plane in Rd\mathbb{R}^d parameterized by WW

    P(Y=1∣x,W)=ϕ(WTx)P(Y=0∣x,W)=1−ϕ(WTx)∵ϕ(a)=11+e−a\begin{aligned} P(Y = 1 \mid x, W) &= \phi (W^T x) \\ P(Y= 0 \mid x, W) &= 1 - \phi (W^T x) \\[12pt] &\because \phi (a) = \frac{1}{1+e^{-a}} \end{aligned}

    maximum likelihood

    1−ϕ(a)=ϕ(−a)1 - \phi (a) = \phi (-a) WML=arg max⁡W∏P(xi,yi∣W)=arg max⁡W∏P(xi,yi,W)P(W)=arg max⁡W∏P(yi∣xi,W)P(xi)=arg max⁡W[∏P(xi)][∏P(yi∣xi,W)]=arg max⁡W∑i=1nlog⁡(τ(yiWTxi))\begin{aligned} W^{\text{ML}} &= \argmax_{W} \prod P(x^i, y^i \mid W) \\ &= \argmax_{W} \prod \frac{P(x^i, y^i, W)}{P(W)} \\ &= \argmax_{W} \prod P(y^i | x^i, W) P(x^i) \\ &= \argmax_{W} \lbrack \prod P(x^i) \rbrack \lbrack \prod P(y^i \mid x^i, W) \rbrack \\ &= \argmax_{W} \sum_{i=1}^{n} \log (\tau (y^i W^T x^i)) \end{aligned}

    equivalent form

    maximize the following:

    ∑i=1n(yilog⁡pi+(1−yi)log⁡(1−pi))\sum_{i=1}^{n} (y^i \log p^i + (1-y^i) \log (1-p^i))
    • url: thoughts/optimization
    • description: softmax

    softmax

    softmax(y)i=eyi∑ieyi\text{softmax(y)}_i = \frac{e^{y_i}}{\sum_{i} e^{y_i}}

    where y∈Rky \in \mathbb{R}^k

    Lien vers l'original
    • url: thoughts/cross-entropy

    cross entropy

    Lien vers l'original

    Vous pourriez aimer ce qui suit

  • ideas.
  • Will
  • Hegel
  • Existentialism
  • geometric projections
  • Open-loop system
  • personal computing
    • Machine learning
    Detects pattern within data and use it to make useful prediction. Generally AI \subset ML \subset DL Some main exploration: Transformers Large language models NLP CNN Logistic regression Optimization gradient descent hyperparameter tuning ensemble learning Recommender systems Reinforcement learning Q-learning Policy Gradient Monte-Carlo Tree Search Generative Models GAN VAE Autoencoder sparse autoencoder sparse crosscoders Supervised Q-learning Low-rank adapters Fields mechanistic interpretability Related: linear algebra.
    Connectionist networks
    and an unstructured overview of patterns in machine learning
    probabilitic modeling
    example: to assume each class is a Gaussian discriminant analysis P(x \mid y = 1, \mu_0, \mu_1, \beta) = \frac{1}{a_0} e^{-\|x-\mu_1\|^2_2} maximum likelihood estimate see also ...