## Multiple Linear Regression - How do we know its the local minimum of the error

guyo13
Posts: 2
Joined: Tue May 16, 2023 8:12 pm

### Multiple Linear Regression - How do we know its the local minimum of the error

Hi,

I took the lecture about the multi dimensional linear model.

I understand that we are trying to minimize the Error function we chose for the model.

What I don't understand are some of the assumptions:
1) That solving the linear system we got by taking the partial derivatives and comparing them to 0 - will yield the weights vector that will specifically minimize the error? How can we assume that the error function has its local minima there (eg. why not local maximum)?

2) How can we assume that X.T.dot(X) is an invertible matrice?

Thanks!
lazyprogrammer
Posts: 127
Joined: Sat Jul 28, 2018 3:46 am

### Re: Multiple Linear Regression - How do we know its the local minimum of the error

1) One way to see is to plot it. Another way to see is to use the methods taught in my latest Calculus course. https://deeplearningcourses.com/c/calculus-data-science

(However it also requires linear algebra knowledge, but I haven't released that course yet)

2) It's not always, and in fact this is discussed in the course!

More prerequisite details will be included in the coming linear algebra course.

However, note that you'll get an error if the matrix is not invertible. Therefore, checking is easy.
guyo13
Posts: 2
Joined: Tue May 16, 2023 8:12 pm

### Re: Multiple Linear Regression - How do we know its the local minimum of the error

Thank you for the amazingly quick response.

Regarding point (2), got it. So at first glance that means that there is a dependence between the training data and whether we can even solve the regression. I will explore this dependence as HW.

Regarding point (1), my curiosity wasn't satisfied - and I'm only after Calculus 2 so only introductory knowledge in 2-variate differential calculus.
I chatted with ChatGPT and there seems to be a "second derivative test" that examines a "Hessian matrix" of the error function and its eigenvalues.

I feel I can explore these methods on my own but I'm curious to what degree do you cover this material in your calculus course?
I'm a CS undergraduate student (and full time software engineer) so I want to make a good time investment if I'm to take it .

Thanks!
lazyprogrammer