Lesson: Inner Products

Question 2

1 point

Transcript — Introduction

You may have not realized it, but you have been using the standard basis vectors for Rn since grade school. In particular, you have been using it ever since you learned that to plot a point (a, b) in the Cartesian plane, that you move a in the x direction and b in the y direction. That is, [a; b] = a times the first standard basis vector plus b times the second standard basis vector. However, in the last lecture, we saw that every n-dimensional vector space is essentially equivalent—that is, isomorphic—to Rn. This means that every n-dimensional vector space must have a basis that is as nice as the standard basis for Rn. Thus, our goal now is to find such nice bases.

To do this, we have to ask what properties does the standard basis have that makes it so easy to use? With a little thought, we realize that two important properties that it has is that all of the vectors are orthogonal to each other, and all of the vectors have unit length. Thus, we want to look for bases in general vector spaces that have these properties. But we realize that we have not yet defined the concepts of orthogonality or length in general vector spaces. In Rn, these are both defined in terms of the dot product. Thus, to get these concepts into general vector spaces, we need to generalize the idea of the dot product to general vector spaces. This will be the goal of this lecture.

Defining Inner Products

The first question we have to ask is how are we going to define this generalization? We recall that to generalize the idea of Rn to a general vector space, we took the important properties of addition and scalar multiplication in Rn and used them to make the definition of a vector space. We now, of course, use the same idea. We will take the three important properties of the dot product and use them to define the general concept called an inner product.

Definition: Let V be a vector space. An inner product on V is a function from (V cross V) to R such that for all vectors v, w, and z in V, and real scalars a and b, we have

  1. The inner product of a vector with itself is always greater than or equal to 0. Also, the inner product of a vector with itself is 0 if and only if the vector is the 0 vector.
  2. The inner product of v and w equals the inner product of w and v.
  3. The inner product of (av + bw) and z equals a times the inner product of v and z plus b times the inner product of w and z.

A vector space with an inner product is called an inner product space.

Notes: A function with property 1 is said to be positive definite. A function with property 2 is called symmetric. A function with property 3 is called left linear. However, notice that since it is left linear and symmetric, it is also right linear. In particular, the inner product of v and (aw + bz) is equal to a times the inner product of v and w plus b times the inner product of v and z. Thus, we say that an inner product is bilinear. Just like a vector space is dependent on which definition of addition and scalar multiplication is being used, an inner product space is dependent on which definition of the inner product is being used. We will see this in the examples.

Inner Products on Rn

Example: The dot product on Rn, which we defined in Linear Algebra I, is an inner product on Rn. Since it is the most commonly used inner product, we call it the standard inner product.

Example: Determine which of the following functions defines an inner product on R3.

(a) The function defined by the inner product of x and y equals 2x1y1 + x2y2 + 3x3y3. Solution: To determine if this is an inner product on R3, we need to check if it satisfies the definition of an inner product. For (1), we have the inner product of x with itself is equal to 2x1-squared + x2-squared + 3x3-squared, which is clearly always greater than or equal to 0. Moreover, the only way this equals 0 is if x1 = x2 = x3 = 0. That is, it equals 0 if and only if x is the 0 vector. Thus, this function is positive definite. For (2), we have the inner product of x and y is equal to 2x1y1 + x2y2 + 3x3y3, which we can rearrange to get this is equal to 2y1x1 + y2x2 + 3y3x3, which, by definition, is equal to the inner product of y and x, and so the function is also symmetric. For (3), the inner product of (ax + by) and z is equal to 2(ax1 + by1)z1 + (ax2 + by2)z2 + 3(ax3 + by3)z3, which can be rewritten as a(2x1z1 + x2z2 + 3x3z3) + b(2y1z1 + y2z2 + 3y3z3), which equals a times the inner product of x and z plus b times the inner product of y and z. Hence, the function is bilinear. Thus, this is an inner product on R3.

(b) The function defined by the inner product of x and y equals x1y1. Solution: This is not an inner product since the inner product of x = [0; 1; 0] with itself gives 0(0), which equals 0. But x is not the 0 vector, so this function is not positive definite, and hence, not an inner product on R3.

(c) The function defined by the inner product of x and y equals (x1-squared)(y1-squared) + (x2-squared)(y2-squared) + (x3-squared)(y3-squared). Solution: The squares in the definition make us think the function is not bilinear. Indeed, the inner product of [2; 0; 0] with itself is (2-squared)(2-squared) + (0-squared)(0-squared) + (0-squared)(0-squared), which equals 16. But 2 times the inner product of [1; 0; 0] and [2; 0; 0] is 2((1-squared)(2-squared) + (0-squared)(0-squared) + (0-squared)(0-squared)), which equals 8. Therefore, the function is not bilinear, and hence is not an inner product.

Note that clearly the inner product on R3 defined in part (a) is different than the dot product. In general, a vector space can have infinitely many different inner products. However, it can be proven that the different inner products do not necessarily differ in an interesting way. For example, in Rn, all inner products behave like the dot product with respect to some basis.

The idea of all of this was to generalize the dot product to other vector spaces, so let’s look at some inner products on other vector spaces.

Inner Products on Other Vector Spaces

Example: On M(2-by-2)(R), we define the inner product of matrices A and B by the trace of (B-transpose)A. Recall that the trace of a matrix is defined to be the sum of the diagonal entries of the matrix. Let’s verify that this is indeed an inner product. Solution: Let A = [a1, a2; a3, a4] and B = [b1, b2; b3, b4] be any 2-by-2 matrices. We have:

  1. The inner product of A with itself equals the trace of (A-transpose)A. We first multiply (A-transpose)A together, except since we are going to take the trace anyways, we only really need to calculate the diagonal entries. Calculating this and then summing the diagonal entries, we get that the inner product of A with itself is a1-squared + a3-squared + a2-squared + a4-squared. This is clearly non-negative, and equals 0 if and only if a1 = a2 = a3 = a4 = 0—that is, if A is the 0 matrix. Thus, the function is positive definite.
  2. We have the inner product of A and B is equal to the trace of (B-transpose)A, which we calculate out to get a1b1 + a2b2 + a3b3 + a4b4. It is easy to verify that this is indeed equal to the trace of (A-transpose)B, which is the inner product of B and A.
  3. Verifying this function is bilinear is tedious, so I’ll skip it. I won’t even recommend it as an exercise. Instead, we look a little more closely at the value of the inner product of A and B that we found in part 2. Notice, it looks very familiar. It is the same as the dot product on the corresponding vectors in R4, [a1; a2; a3; a4] and [b1; b2; b3; b4]. That is, it is equivalent to the dot product. Thus, we will call the inner product of A and B equals the trace of (B-transpose)A the standard inner product on M(m-by-n)(R).

Note: Whenever calculating values of this inner product, we never actually calculate the trace of (B-transpose)A. We instead mimic the dot product formula by summing the products of the corresponding entries.

Example: Let A = [1, 3; 4, -1] and B = [2, 1; -1, -1] be matrices in M(2-by-2)(R). Calculate the inner product of A and B under the standard inner product for M(2-by-2)(R). Solution: Using our work above, we get that the inner product of A and B is 1(2) + 3(1) + 4(-1) + (-1)(-1), which equals 2.

Note: Unless we specify otherwise, whenever we say Rn or M(m-by-n)(R), we always mean the inner product space under the standard inner product.

Example: On P2(R), the function defined by the inner product of p(x) and q(x) equals p(0)q(0) + p(1)q(1) + p(2)q(2) is an inner product. I will leave the verification that this is an inner product on P2(R) as an exercise. Note, once again, that proving this function is bilinear is tedious.

Example: Find the inner product of x and (x-squared + x + 1) using the inner product of P2(R) defined above. Solution: In this case, we are using p(x) = x and q(x) = x-squared + x + 1. Thus, we get the inner product of p(x) and q(x) equals p(0)q(0) + p(1)q(1) + p(2)q(2). Evaluating each of these polynomials at the given values of x gives that this is equal to 0(1) + 1(3) + 2(7), which equals 17.

Example: On the vector space C[a, b] of continuous functions defined on the interval from a to b, we define a very important inner product that is an integral part of applied mathematics. We define the inner product of f(x) and g(x) to be the integral from a to b of f(x)g(x) dx. This inner product is the basis (no pun intended) for Fourier series.

Example: In C[-pi to pi], find the inner product of 1 and x under the inner product defined above. Solution: We get the inner product is the integral from –pi to pi of 1(x) dx. The antiderivative of x is (1/2)x-squared, so we get the inner product is (1/2)(pi-squared) – (1/2)((-pi)-squared), which equals 0. Recall, in Linear Algebra I, we said that if two vectors had dot product equal to 0, then they were orthogonal. Does that mean 1 and x are orthogonal? Yes—under this inner product.

In the next lecture, we will look at the concepts of orthogonality and length in general inner product spaces.

© University of Waterloo and others, Powered by Maplesoft