> Would it be helpful to actually study a lot of cryptography before I start a job?
I would definitely say yes. There's almost a sort of stereotype for mathematicians who try to design cryptography without understanding it and making beginner mistakes.
I'd suggest at least going through something like Dan Boneh's Crypto I class, which is free online via Coursera:
Given your background you might want to pay particular attention to things like RSA and El Gamal. Unfortunately the class doesn't cover things like ECC and Lattices which you might like, nor do I have a similar "crash course" resource for those.
There are definitely a lot of jobs in theoretical cryptography out there. However, many of them are in the cryptocurrency space, which may not be your cup of tea.
Beyond that, you might look at large tech companies that have a dedicated cryptography team (e.g. Google)