Implement Dot Product

Linear Algebra

Easy

Problem

Theory

The dot product takes two vectors of the same length and returns a single number. It is one of the most fundamental operations in linear algebra and appears in nearly every machine learning algorithm.

For two vectors x and y of length n:

x \cdot y = \sum_{i=1}^{n} x_i y_i = x_1 y_1 + x_2 y_2 + \cdots + x_n y_n

You multiply corresponding elements and sum the results.

A Simple Example

Let x = [1, 2, 3] and y = [4, 5, 6].

x \cdot y = (1)(4) + (2)(5) + (3)(6) = 4 + 10 + 18 = 32

The result is a scalar (single number), not a vector. This is why the dot product is also called the scalar product.

The Geometric Interpretation

The dot product has a beautiful geometric meaning:

x \cdot y = ||x|| \cdot ||y|| \cdot \cos(\theta)

Where:

• ||x|| is the length (magnitude) of vector x
• ||y|| is the length of vector y
• \theta is the angle between the two vectors

This formula connects algebra to geometry. The dot product tells you:

• How aligned the vectors are (via \cos\theta)
• How long each vector is (via the magnitudes)

What the Sign Tells You

The dot product can be positive, negative, or zero:

Positive dot product (x \cdot y > 0):

• The angle \theta is acute (less than 90 degrees)
• The vectors point in roughly the same direction

Zero dot product (x \cdot y = 0):

• The angle \theta is exactly 90 degrees
• The vectors are orthogonal (perpendicular)
• This is extremely important in many applications

Negative dot product (x \cdot y < 0):

• The angle \theta is obtuse (greater than 90 degrees)
• The vectors point in roughly opposite directions

The Dot Product and Vector Length

The dot product of a vector with itself gives the squared length:

x \cdot x = ||x||^2 = x_1^2 + x_2^2 + \cdots + x_n^2

So the length is:

||x|| = \sqrt{x \cdot x}

This is the Euclidean norm, the most common way to measure vector magnitude.

Projection: Decomposing One Vector Along Another

One of the most useful applications of the dot product is projection. The projection of x onto y tells you "how much of x points in the direction of y."

Scalar projection (length of the shadow):

\text{comp}_y x = \frac{x \cdot y}{||y||}

Vector projection (the actual vector in the direction of y):

\text{proj}_y x = \frac{x \cdot y}{||y||^2} y = \frac{x \cdot y}{y \cdot y} y = \text{comp}_y x \cdot \frac{y}{||y||}

This decomposition is the foundation of least squares, Gram-Schmidt orthogonalization, and many other algorithms.

Dot Product in Matrix Notation

Using matrix notation, the dot product of column vectors x and y is:

x \cdot y = x^T y

Here, x^T is a 1 \times n row vector, y is an n \times 1 column vector, and the result is a 1 \times 1 matrix (a scalar).

This notation connects dot products to matrix multiplication: the entry (i, j) of the matrix product AB is the dot product of row i of A with column j of B.

Where Dot Products Appear in Machine Learning

Linear models:

A linear model predicts \hat{y} = w^T x + b. The term w^T x is a dot product between the weights w and the features x. The prediction is literally "how aligned is this input with the learned weight vector?"

Neural networks:

Each neuron computes a weighted sum of its inputs: z = w^T x + b. This is a dot product plus a bias. The entire forward pass of a neural network is built from dot products.

Cosine similarity:

To compare two vectors while ignoring their magnitudes, normalize them first:

\text{cosine similarity} = \frac{x \cdot y}{||x|| \cdot ||y||}

This is the cosine of the angle between them. It is 1 for identical directions, 0 for perpendicular, and -1 for opposite.

Attention mechanisms:

In transformers, attention scores are computed using dot products between query and key vectors:

\text{score}(q, k) = q^T k

This measures how relevant each key is to the query.

Properties of the Dot Product

Commutative:

x \cdot y = y \cdot x

Distributive:

x \cdot (y + z) = x \cdot y + x \cdot z

Scalar multiplication:

(cx) \cdot y = c(x \cdot y) = x \cdot (cy)

Non-negative self-dot:

x \cdot x \geq 0 with equality only when x = 0.

Computational Considerations

The dot product of two length-n vectors requires:

• n multiplications
• n - 1 additions
• Total: O(n) operations

For very long vectors, the order of summation can affect numerical precision due to floating-point errors. Summing from smallest to largest magnitudes is often more accurate.

In highly optimized code, dot products use SIMD (Single Instruction Multiple Data) to process multiple elements in parallel, making them very fast on modern hardware.

The Cauchy-Schwarz Inequality

One of the most important inequalities in mathematics involves the dot product:

|x \cdot y| \leq ||x|| \cdot ||y||

Equality holds when x and y are parallel (one is a scalar multiple of the other).

This inequality is the foundation of many proofs in linear algebra and functional analysis. It guarantees that the cosine similarity is always between -1 and 1.

Examples

Hints

1. Convert both vectors with np.asarray(..., dtype=float).
2. Use np.dot(x, y) and convert the NumPy scalar to float.

Requirements

• Compute the dot product with NumPy
• Return a Python float

Constraints

• x and y are equal-length, nonempty one-dimensional numeric lists
• Each vector contains at most 10^6 values
• Use NumPy only

Starter Code

import numpy as np

def dot_product(x: list, y: list) -> float:
    """
    Returns the dot product as a float.
    """
    # Write code here
    pass

Test Cases