I have been pulling my hair trying to understand how AI works. With some very basic understanding of the math behind it, I know it's all linear equations and such. Math is not the question here as I am not trying to solve math. What's really bothering me is how you teach a computer to self-learn. I mean, developers write code so that computers can follow these instructions, right? How does a computer go from "follow instructions" to "think, learn an now generate content"? What kind of instructions do you write code so that they can learn??? Trying to get an answer from people around me, but haven't been able to find a convincing answer.
At its core, all that's happening is similar to polynomial approximation. Except in much higher dimensions. That's all there is to it, mathematically speaking.
You must be the parent to a really really smart 5 year old :D
AI sees patterns mostly. Start with the most basic algorithm âLinear regressionâ. If you see 10 houses with 1 bedroom apartment being sold for 100k 2 bedroom for 200k and 10 bedroom apartment for 1 million dollars. You see that 100k times the bedroom is an indicator of the selling price of the house. In reality however there are many more independent variables apart from the # of bedrooms and often times not enough. Also the relationship will never be this simple and linear. AI helps us identify non linear relationship between multidimensional independent variables. Itâs a little abstract but you canât explain AI in a single post.
AI is a catch all phrase. I think you mean what genai or things like chatGPT are. These ai functions mimic words, much like a human child. Over time and learning (just like adults teach kids), the system is able to output first word and predict the most sensible next word, and so on. Over time, words become phrases and system can put a best possible phrase and next phrase with feedback from users. The difference between child and genai however is that genai has no contructs for âself awarenessâ or emotions. At least not in a traditional sense. If youâve heard about AGI- thatâs the problem it will solve where such genai will have some sense of self awareness and emotional intelligence.
Great explanation. Would you agree that the current crop of LLMs are not on an inevitable path to AGI? LLMs are like the faster horse compared to neural nets. But AGI is like a rocket.
If
The current batch of genAI systems donât learn. They are pretrained (the P in GPT) to understand all the word/token associations necessary to predict the next most likely word/token in the output. It isnât remotely like actual reasoning/problem solving. There is no cognition / metacognition / learning to learn⌠There are older AI/ML solutions that are more online learners, but they are not capable of the kinds of generative tasks we are seeing from genAI right now. Explained like I would to a 5 yr old: AI isnât smart, it is just pretending to be smart
When you ask it to convert java code to say, python code, that cant simply be next word prediction
@Microsoft: It is, I promise. It has embedded in the transformer model all the associations btw the elements of your Java tokens and then has probabilistic mappings to python tokens gleaned from GitHub, stack overflow, etc. This is why these models have to be trained over trillions of tokens from the entire internet â theyâre just repeating back (probabilistically) things theyâve seen. LLMs donât have deep understanding and are often wrong in these kinds of tasks, requiring human editing. Plus, this is a trivial task - basically a fancy translation function. If you want to see real failure to learn and reason, ask simple logic questions like âAlice has two brothers and three sisters, how many sisters does Aliceâs brother have?â - they canât world build or infer gender/sex from name w/o being explicitly told - then theyâll get it right in that specific case, but fail if you vary the problem slightly (something a 5 yr old would recognize).
Pytorch tutorials. Also remember in high school, getting a lot of data points and making a line of best fit? AI does that exact same thing, but it has like millions of those graphs.
Basically it tries to predict your next word based on how likely you are to use that next word
AI is a generic term. Do you mean LLMs? Because they are hot right now.
Throw poop at the wall and see what sticks
Did you try asking chatgpt?