I’m working on general infra or cloud stuff and would like to make a switch to AI/ML infra track. So what’s the difficulty to make the move in this brutal job market? I heard many AI startup recruiting due to openAI success, what about AI infra opportunities?
Tbh, if you are a solid infra person ai/ml infra ain’t that much different. With gen ai, focus is more on inference than training. Inference: big models that need to be stored in memory and weights need to be kept secure because that’s the secret sauce. Training: long times think months with big cluster of Gpu’s If you can mess around renting couple of non high end (a100/a10/h100) are the ones in demand. Finetune a small model gpt2 on an instruct dataset(you should be able to find code/weights/data) in a multi node setup using kubernetes that’s more than enough
Thanks for the insightful comment! Yes I agree general infra not much diff from ml infra esp for serving. But I’m asking for interview/job hop. Given the brutal job market now, it’s likely company will reject me because I don’t have exactly hands-on ML infra experience
Linkedin has ml teams. See if you can switch internally. This may sound like “duh” but they aren’t rejecting you. They are just accepting some infra who claims they have ml experience and parrots definitions. Mle’s and ml infra folks (in my anecdotal experience) only know ml terms. Don’t really understand ml. And mle are worse at se than general se and ml infra are less competitive at infra than general infra folks
Op, I’m also a data infra engineer … we can sync up and look for AI infra roles
same boat here. Wondering where you guys end up applying or joining
Engage yourself in MLOps structure you can use your existing infrastructure knowledge along with learning new things like model governance.
AI infra is critical for a successful ML data machine. If you can prove you’re good here you’ll be as sought after as a strong MLE
I don’t have ML/modeling experience at industry. So I’m asking for ML infra
That’s okay. I’m saying if you’re good at ML infra you’re just as important as a “fancy” MLE role. They are equally required for a successful ML based company