Gradient Hacking Gemma 4 hallucination

<details>
<summary><b>Quick Summary: What is Gradient Hacking?</b></summary>
Gradient hacking is a broad term describing the manipulation of the gradients used during the training of a neural network to either <b>attack</b> the model (adversarial attacks) or <b>exploit</b> the training process to achieve a specific, often hidden, goal (such as embedding backdoors).
</details>

In deep learning, the "gradient" refers to the partial derivative of the loss function with respect to the model's weights. This gradient tells the model how to adjust its parameters to minimize error via **Stochastic Gradient Descent (SGD)**.

Gradient hacking occurs when an actor manipulates these mathematical signals. This can be categorized into three primary domains:

### 1. Adversarial Attacks (Inference Time)
This is the most common form of "hacking" the gradient. Instead of updating the model's weights to improve performance, the attacker calculates the gradient to find the smallest possible change to the **input data** that will cause the model to misclassify the input.

The most famous method is the **Fast Gradient Sign Method (FGSM)**. The goal is to maximize the loss $J(\theta, x, y)$ by moving the input $x$ in the direction of the gradient:

$$x_{adv} = x + \epsilon \cdot \text{sign}(\nabla_x J(\theta, x, y))$$

Where:
*   $x_{adv}$ is the adversarial example.
*   $\epsilon$ is a small scalar controlling the magnitude of the perturbation.
*   $\nabla_x$ is the gradient with respect to the input.

### 2. Poisoning and Backdoor Attacks (Training Time)
In this scenario, the "hack" happens during the training phase. An attacker injects a small amount of "poisoned" data into the training set. 

By carefully crafting the labels and features of this data, the attacker ensures that the gradients during training push the model toward a state where it performs perfectly on normal data but triggers a specific behavior (a "backdoor") when a specific trigger (e.g., a small white square in the corner of an image) is present.

### 3. Gradient Leakage (Privacy Attacks)
In **Federated Learning**, multiple clients train a model locally and send only their gradients to a central server. "Gradient hacking" here refers to **Gradient Inversion**.

An attacker with access to the gradients $\nabla W$ can mathematically reconstruct the original private training images or text. Because the gradient is calculated as:

$$\nabla W = \frac{\partial \text{Loss}}{\partial W}$$

The attacker can treat the original input $x$ as a variable and optimize a dummy input $x'$ until the gradient produced by $x'$ matches the observed gradient $\nabla W$.

---

### Comparison Table: Types of Gradient Manipulation

| Type | Target | Objective | Timing |
| :--- | :--- | :--- | :--- |
| **Adversarial** | Input Data | Force misclassification | Inference |
| **Poisoning** | Model Weights | Embed hidden triggers | Training |
| **Inversion** | Private Data | Steal training samples | Communication |

### How to Defend Against Gradient Hacking
To prevent these vulnerabilities, researchers use several techniques:

*   **Adversarial Training:** Including adversarial examples in the training set so the model learns to ignore the "noise."
*   **Gradient Clipping:** Limiting the maximum value of a gradient to prevent extreme updates (common in RNNs).
*   **Differential Privacy:** Adding random noise to the gradients before they are shared in federated learning to prevent inversion.
*   **Weight Decay/Regularization:** Using $\ell_2$ regularization to prevent the model from becoming overly sensitive to small input changes:
    $$\text{Loss}_{total} = \text{Loss}_{data} + \lambda \sum ||w||^2$$