How to get a specific machine type for ML Engine online prediction?

Question

Is there an option to request a faster node for online prediction in ML Engine?

For example, when training I can configure any of these machines for my job:

standard, large_model, complex_model_s, complex_model_m, complex_model_l, standard_gpu, complex_model_m_gpu, complex_model_l_gpu, standard_p100, complex_model_m_p100

See description of available clusters and machines for training here and here

I am struggling to find if it is possible to control what kind of machine runs my online prediction.

Answer 1

We are currently adding that capability and will let you know when it's publicly available.

Answer 2

ML Engine offers 4-core instance type in addition to the default serving instance type for online prediction. However the feature is still at alpha stage and it will only be available to a selected list of accounts who opted in as "Trusted Testers". Please contact [email protected] if you need help to setup prediction service with faster node.