Now a bit of background on myself, I am a RS with a PhD and 7 YOE and did quite a lot of research on NLP with top researchers in the field. I have never seen anything quite impressive as ChatGPT. Many companies tried to create very large Language models with billions of parameters. They all have the same problem. They are overfitting the sequences they are trained on. They just remember the context and try to generate a plausible answer/sentence. After a few prompts, it's easy to spot problems/weaknesses. Now with ChatGPT, it's different. It's not perfect by any means but it's the largest leap I have ever seen. it can solve ML problems that usually need separate training data and special fine-tuning. For instance, try to ask for a recommendation for a song given that you like a particular genre or an artist. That's a very specific Recommendation systems problem that companies like Spotify are trying to solve. Similar goes with movies (watch out Netflix). I am not sure how they got there. I know at the high level they are using transformers with RL fine tuning (Many have tried that before) but this does not explain how it manages to get these tasks right. Their reward function must very interesting My prediction is that the first person to try to solve ML problems using Text data is going to open a whole lot of possibilities. A simple example is to convert the Iris dataset to text format describing every row of the dataset. Then train a LLM on this data and try to generate predictions through this Language model to determine the flower type. see how this compares to traditional ML. Think about it. Humans do not learn from tabular data but rather from signals (Image, text, Audio, Video). ChatGPT learned these tasks from text signals only. Combining this with Images, and maybe videos will provide a new push to the boundary of AI. Google is right to sound the alarm. I think many companies should do so. Any company where the main business involves text data is doomed in the near future. Think Grammarly for instance. I have just asked ChatGPT to correct an email and it has done a much better job than theirs. I wish I joined OpenAI when I had the chance :( TC 700K (I think, I haven't checked my grant for a while)
Will ML engineers be out of jobs soon. Chatgpt will do your jobs.
If you think ML engineers will be the first to go then you don’t know what an MLE actually does
Accenture isn't wrong. We have tools like SQL, so need fewer hardcore database engineers in the industry. Similarly, MLE roles will be cannibalized by prompt engineering. It will still exist, but comparable to niche roles for coding in assembly language.
Grammarly is 100% doomed
Guess even ML scientists can have stupid takes...
Interesting perspective! Yeah the biggest advantage of natural language interface is its ease to express things. That is the API between humans which makes us really good at using it. Any system built with NL as the main interface will explode if it is doing a good job. A search engine is a good example. Before search engine became common way to interact with web there were web portal services that were trying to categorize contents and provide hierarchical browsing experience, very much like encyclopedia or library. But search engine completely beat it; if you can do natural language interface you should! :)
You nailed it.
A search engine isn't quite using natural language though. There's an implicit "language" there.
OP but what about accuracy and trust? I'm not sure if you've played around with it enough but it has gotten many things totally wrong for me in subtle ways. Try asking it about the things that you're an expert at. Also even Google does recommendations if you type "films like xyz." In fact if you compare those results and ChatGPT's results you won't be so impressed with the latter. It's obvious that Google is already sitting on a better system that it only reveals in parts lest it cannibalizes its own business.
True. It's far from perfect. But much better than any similar model Don't forget Google is also relying on web crawlers. Not the same thing. We can't compare a language model to a search engine. If anything it's more scary. If chatGPT got close to Google search engine performance without crawling the web. That's really cool
Worst even, when you tell it that it’s wrong and correct it’s mistakes then ask if it will learn from this it will tell you that it’s unable to learn and it’s patterns are all static
Imma ask chatGPT to tell me the difference between an iris setosa and an iris pakora. Let's see if it knows.
Dude, it’s ground breaking but it is not mind shattering. They had shit ton of data and used transformers which is exactly what transformers excel at.
Your prediction has already happened. Some student got a conference poster out of it ( he was from Carnegie Mellon iirc). The results were very promising.
Is it really that much better than the raw model plus a bunch of fine tuning? I can get shockingly good responses from code-davinci-2 and a good prompt on a wide variety of tasks without even doing any fine tuning.
It is better than a human engjneer
Written by ChatGPT
OP what was ur prompt?