The ML Engineer vs the MLOps persona

Reading Time: 3 minutes

Introduction

As enterprises across the globe step up AI/ML adoption rates, there is an increasing need to have mature processes for smooth AI/ML operations. There is a need for enterprises to clearly define the roles and responsibilities for all stakeholders in the AI/ML operation lifecycle. It is very common for enterprises to have some level of ambiguity regarding the roles and responsibilities of the various stakeholders in the ecosystem. The most common roles in the AI/ML ecosystem are the Data Scientist, Model Validator, Machine Learning Engineer, MLOps engineer, etc. In this blog, we demystify a Machine Learning Engineer’s roles and responsibilities and how they differ for an MLOps persona.

We will cover the following topics in this article:

Engineering vs operations in general

  • What is ML engineering? What are the roles and responsibilities of an ML Engineer?
  • What is MLOps? What are the roles and responsibilities of the MLOps persona?
  • What are the challenges faced by an ML Engineer?
  • What are the challenges faced by the MLOps persona?
  • How does the ML Engineer compare with the MLOps persona?
  • How does Fosfor(Refract) help the ML Engineer?
  • How does Fosfor(Refract) helps the MLOps persona?

Engineering vs operations in general

For any product or service to succeed in the market, the offering enterprise should have a strong capability in engineering and operations.

The following is a brief comparison of focus areas for both these functions:

Focuses on innovation. Needs strong technical skills.Focuses on automation. Needs strong automation skills.

 Engineering  Operations
 Focuses on building products/services that are highly   scalable and highly performant.  Focuses on delivering products/services in production and ensuring   service  quality is always maintained.
 Focuses on providing permanent fixes in response to any   incident/ bug.  Focuses on ensuring product/service is up and running.
 Not an end-user-facing role.  End-user-facing role that requires to have strong communication skills.

What is ML engineering? What are the roles and responsibilities of an ML Engineer?

A Machine Learning engineer (ML engineer) is a stakeholder in the ML lifecycle who works on the research, build, and design of self-running Artificial Intelligence (AI) systems for predictive modeling. An ML Engineer’s primary goals include creating machine learning models and retraining systems when needed. Although responsibilities may differ depending on the organization, some typical duties for this role include:

  • Building ML model training pipelines.
  • Building ML inference pipelines.
  • Integrating models with external applications/API gateways.
  • Building CI/CD pipelines for deploying the models to higher environments.
  • Controlling the model versions in the development environment.
  • Ensuring model robustness in terms of scalability and performance.

Challenges faced by an ML Engineer:

  • ML Engineers spend a lot of time building training pipelines and inferencing pipelines. After they write some script or use Python inbuilt packages for building the pipelines, they would have to again write shell script, etc., for automating that pipeline, which is not the best use of their time.
  • ML Engineers spend a lot of time packaging all the components of the model for shipping to higher environments.
  • ML Engineers spend a lot of time creating APIs for the model, which is not an efficient use of their time.
  • ML Engineers need to generate synthetic data for training in case of an imbalanced dataset.

What is MLOPS? What are the roles and responsibilities of the MLOps persona?

MLOps is an integral part of Machine Learning that deploys and maintains ML models in production reliably and efficiently. The MLOps persona seeks to increase automation and improve the quality of production models while also focusing on business and regulatory requirements.

The following are some of the roles and responsibilities of the MLOps persona:

  • Deploy the models from QA to the production environment.
  • Control model versioning in the production environment.
  • Monitor the model in production for feature drift, performance drift, prediction drift, and label drift.
  • Monitor the model in production for bias.
  • Monitor the model service health in production.
  • Monitor the model resource consumption in production.
  • Ensuring models can scale up and down as and when traffic increases or reduces.

Challenges faced by the MLOps persona:

  • The MLOps persona must spend a lot of time calculating feature, performance, label, and prediction drifts.
  • The MLOps persona does not get alerts regarding the performance degradation of the model.
  • The MLOps persona will have challenges getting the details of the resource consumption in real-time.
  • The MLOps persona will have difficulties managing version controls of the models in production.

Comparison between the ML Engineer and the MLOps persona

 ML Engineer  MLOps
 Works closely with Data Scientists and the Model   Validation team.  Works closely with the business owners of the   model.
 Builds training pipelines and inferencing pipelines.  Automates training pipelines and inferencing pipelines in   production.
 Builds CI/CD pipelines for moving the code to a   higher environment.  Utilizes CI/CD pipelines for deploying the models  in   production.
 Model validation for performance factors like   accuracy, precision,   etc. in development.  Model monitoring for feature, prediction, label, and   performance drifts.
 Model version control in development /QA.  Model version control in production.
 Responsible for building integration with other applications in the   development environment.  Responsible for applying integrations with other components   in the production environment.
 Success is measured by metrics such as the number of  defects that   occurred in higher environments.  Success is measured by metrics such as the number of   incidents resolved in production.

How Refract can help the ML Engineer

Refract offers a variety of benefits for the ML engineer. It helps automate various aspects of the machine learning lifecycle, such as data preparation, model training, model deployment, and monitoring. By automating these tasks in an ML pipeline, Refract can also help improve the quality and reliability of the machine learning system by making it easier to debug, test, and optimize the models.

Additionally, Refract can help improve communication and collaboration between team members working on the ML project, such as Data Scientists, ML engineers, and IT engineers. This can lead to better coordination, faster development cycles, and more efficient use of resources.

Refract offers the following features as out-of-the-box capabilities specifically to aid the ML Engineer:

  • SDK for data extraction which helps in automation of the process.
  • Workflow orchestration for building training pipelines and inferencing pipelines.
  • Model version control.
  • Build-time metrics for validating the models’ performance.
  • Model registration and model deployment.
  • Model API.
  • Workflow for bulk scoring.
  • Scheduler for scheduling based on time or event trigger.

How Refract can help the MLOps persona

Refract can help automate the process of building, testing, and deploying models, making it easier to manage large numbers of models and track their performance over time. Refract offers the following features as out-of-the-box capabilities specifically to aid the MLOpspersona:

  • Models developed on other platforms can be deployed in Refract for monitoring.
  • Automated alerts based on threshold values for feature, performance, prediction, and label drifts.
  • Automated alerts on successful completion or failure of a scheduled job.
  • Automated alerts on a service outage.
  • Resource utilization metrics for the model over a period.
  • Build time metrics for validating model performance.
  • Model registration and model deployment.
  • Model API.
  • Workflow for bulk scoring.
  • Scheduler for scheduling based on time or event trigger.

Conclusion

The roles of a Machine Learning Engineer (ML Engineer) and the MLOps persona are essential components of the AI/ML ecosystem, each contributing distinct responsibilities in the development and maintenance of machine learning models. While ML Engineers are primarily focused on model creation and training, MLOps personnel take charge of deploying models in production and ensuring their reliability.

Both roles face unique challenges, with ML Engineers grappling with time-consuming tasks like pipeline development and API creation, and MLOps personas dealing with the complexities of monitoring, drift detection, and resource management in production environments. Understanding these distinctions is vital for organizations to effectively streamline their AI/ML operations.

Refract offers valuable solutions to address these challenges, aiding ML Engineers with automation, version control, and performance metrics, while also empowering MLOps professionals with comprehensive model monitoring and automated alerts. With the right tools and a clear understanding of their roles, both ML Engineers and MLOps personnel can contribute to the successful deployment and maintenance of machine learning models in today’s AI-driven landscape.

Author

Ravikumar S Haligode

Senior Specialist – Data Science, Fosfor

With over 15 years of IT experience, Ravikumar has worked closely with senior stakeholders from business, operations, and system owners to identify opportunities for cost reduction, revenue enhancement, and customer experience using a data-driven approach. He has worked on multiple AI/ML projects, with extensive experience in building and evaluating models, tuning hyperparameters for optimum performance, and retraining models.

More on the topic

Read more thought leadership from our team of experts

How to choose the best AI/ML platform for your business

Although according to a 2020 McKinsey study1, 50% of the companies surveyed had already adopted AI in at least one business function, the state of AI in 2023 according to a similar McKinsey study suggests that adoption rates have effectively plateaued over the last 3 years2.

Read more

Accelerate your production ML journey with Refract

As we all know, production ML (Machine Learning) is more engineering than machine learning. Building a prototype in machine learning has become very simple nowadays, all thanks to different open-source projects like sci-kit, TensorFlow, Keras, etc. But operationalizing that model to get the insights from the model which can be used in day-to-day business decisions is challenging and needs more engineering knowledge than data science knowledge.

Read more

AI in a box: How Refract simplifies end-to-end machine learning

The modern tech world has become a data hub reliant on processing. Today, there is user data on everything from driving records to scroll speed on social media applications. As a result, there has been a considerable demand for methods to process this data, given that it holds hidden insights that can propel a company into the global stage quicker than ever before.

Read more
We use cookies to personalise content and ads, to provide social media features and to analyse our traffic. We also share information about your use of our site with our social media, advertising and analytics partners. View more
Cookies settings
Accept
Privacy & Cookie policy
Privacy & Cookies policy
Cookie name Active

What is a cookie?

A cookie is a small piece of data that a website asks your browser to store on your computer or mobile device. The cookie allows the website to “remember” your actions or preferences over time. On future visits, this data is then returned to that website to help identify you and your site preferences. Our websites and mobile sites use cookies to give you the best online experience. Most Internet browsers support cookies; however, users can set their browsers to decline certain types of cookies or specific cookies. Further, users can delete cookies at any time.

Why do we use cookies?

We use cookies to learn how you interact with our content and to improve your experience when visiting our website(s). For example, some cookies remember your language or preferences so that you do not have to repeatedly make these choices when you visit one of our websites.

What kind of cookies do we use?

We use the following categories of cookie:

Category 1: Strictly Necessary Cookies

Strictly necessary cookies are those that are essential for our sites to work in the way you have requested. Although many of our sites are open, that is, they do not require registration; we may use strictly necessary cookies to control access to some of our community sites, whitepapers or online events such as webinars; as well as to maintain your session during a single visit. These cookies will need to reset on your browser each time you register or log in to a gated area. If you block these cookies entirely, you may not be able to access gated areas. We may also offer you the choice of a persistent cookie to recognize you as you return to one of our gated sites. If you choose not to use this “remember me” function, you will simply need to log in each time you return.
Cookie Name Domain / Associated Domain / Third-Party Service Description Retention period
__cfduid Cloudflare Cookie associated with sites using CloudFlare, used to speed up page load times 1 Year
lidc linkedin.com his is a Microsoft MSN 1st party cookie that ensures the proper functioning of this website. 1 Day
PHPSESSID ltimindtree.com Cookies named PHPSESSID only contain a reference to a session stored on the web server When the browsing session ends
catAccCookies ltimindtree.com Cookie set by the UK cookie consent plugin to record that you accept the fact that the site uses cookies. 29 Days
AWSELB Used to distribute traffic to the website on several servers in order to optimise response times. 2437 Days
JSESSIONID linkedin.com Preserves users states across page requests. 334,416 Days
checkForPermission bidr.io Determines whether the visitor has accepted the cookie consent box. 1 Day
VISITOR_INFO1_LIVE Tries to estimate users bandwidth on the pages with integrated YouTube videos. 179 Days
.avia-table-1 td:nth-of-type(1):before { content: 'Cookie Name'; } .avia-table-1 td:nth-of-type(2):before { content: 'Domain / Associated Domain / Third-Party Service'; } .avia-table-1 td:nth-of-type(3):before { content: 'Description'; } .avia-table-1 td:nth-of-type(4):before { content: 'Retention period'; }

Category 2: Performance Cookies

Performance cookies, often called analytics cookies, collect data from visitors to our sites on a unique, but anonymous basis. The results are reported to us as aggregate numbers and trends. LTI allows third-parties to set performance cookies. We rely on reports to understand our audiences, and improve how our websites work. We use Google Analytics, a web analytics service provided by Google, Inc. (“Google”), which in turn uses performance cookies. Information generated by the cookies about your use of our website will be transmitted to and stored by Google on servers Worldwide. The IP-address, which your browser conveys within the scope of Google Analytics, will not be associated with any other data held by Google. You may refuse the use of cookies by selecting the appropriate settings on your browser. However, you have to note that if you do this, you may not be able to use the full functionality of our website. You can also opt-out from being tracked by Google Analytics from any future instances, by downloading and installing Google Analytics Opt-out Browser Add-on for your current web browser: https://tools.google.com/dlpage/gaoptout & cookiechoices.org and privacy.google.com/businesses
Cookie Name Domain / Associated Domain / Third-Party Service Description Retention period
_ga ltimindtree.com Used to identify unique users. Registers a unique ID that is used to generate statistical data on how the visitor uses the web site. 2 years
_gid ltimindtree.com This cookie name is asssociated with Google Universal Analytics. This appears to be a new cookie and as of Spring 2017 no information is available from Google. It appears to store and update a unique value for each page visited. 1 day
_gat ltimindtree.com Used by Google Analytics to throttle request rate 1 Day
.avia-table-2 td:nth-of-type(1):before { content: 'Cookie Name'; } .avia-table-2 td:nth-of-type(2):before { content: 'Domain / Associated Domain / Third-Party Service'; } .avia-table-2 td:nth-of-type(3):before { content: 'Description'; } .avia-table-2 td:nth-of-type(4):before { content: 'Retention period'; }

Category 3: Functionality Cookies

We may use site performance cookies to remember your preferences for operational settings on our websites, so as to save you the trouble to reset the preferences every time you visit. For example, the cookie may recognize optimum video streaming speeds, or volume settings, or the order in which you look at comments to a posting on one of our forums. These cookies do not identify you as an individual and we don’t associate the resulting information with a cookie that does.
Cookie Name Domain / Associated Domain / Third-Party Service Description Retention period
lang ads.linkedin.com Set by LinkedIn when a webpage contains an embedded “Follow us” panel. Preference cookies enable a website to remember information that changes the way the website behaves or looks, like your preferred language or the region that you are in. When the browsing session ends
lang linkedin.com In most cases it will likely be used to store language preferences, potentially to serve up content in the stored language. When the browsing session ends
YSC Registers a unique ID to keep statistics of what videos from Youtube the user has seen. 2,488,902 Days
.avia-table-3 td:nth-of-type(1):before { content: 'Cookie Name'; } .avia-table-3 td:nth-of-type(2):before { content: 'Domain / Associated Domain / Third-Party Service'; } .avia-table-3 td:nth-of-type(3):before { content: 'Description'; } .avia-table-3 td:nth-of-type(4):before { content: 'Retention period'; }

Category 4: Social Media Cookies

If you use social media or other third-party credentials to log in to our sites, then that other organization may set a cookie that allows that company to recognize you. The social media organization may use that cookie for its own purposes. The Social Media Organization may also show you ads and content from us when you visit its websites.

Ref links:

LinkedInhttps://www.linkedin.com/legal/privacy-policy Twitterhttps://gdpr.twitter.com/en.html & https://twitter.com/en/privacy & https://help.twitter.com/en/rules-and-policies/twitter-cookies Facebookhttps://www.facebook.com/business/gdpr Also, if you use a social media-sharing button or widget on one of our sites, the social network that created the button will record your action for its own purposes. Please read through each social media organization’s privacy and data protection policy to understand its use of its cookies and the tracking from our sites, and also how to control such cookies and buttons.

Category 5: Targeting/Advertising Cookies

We use tracking and targeting cookies, or ask other companies to do so on our behalf, to send you emails and show you online advertising, which meet your business and professional interests. If you have registered on our websites, we may send you emails, tailored to reflect the interests you have shown during your visits. We ask third-party advertising platforms and technology companies to show you our ads after you leave our sites (retargeting technology). This technology allows us to make our website services more interesting for you. Retargeting cookies are used to record anonymized movement patterns on a website. These patterns are used to tailor banner advertisements to your interests. The data used for retargeting is completely anonymous, and is only used for statistical analysis. No personal data is stored, and the use of the retargeting technology is subject to the applicable statutory data protection regulations. We also work with companies to reach people who have not visited our sites. These companies do not identify you as an individual, instead rely on a variety of other data to show you advertisements, for example, behavior across websites, information about individual devices, and, in some cases, IP addresses. Please refer below table to understand how these third-party websites collect and use information on our behalf and read more about their opt out options.
Cookie Name Domain / Associated Domain / Third-Party Service Description Retention period
BizoID ads.linkedin.com These cookies are used to deliver adverts more relevant to you and your interests 183 days
iuuid demandbase.com Used to measure the performance and optimization of Demandbase data and reporting 2 years
IDE doubleclick.net This cookie carries out information about how the end user uses the website and any advertising that the end user may have seen before visiting the said website. 2,903,481 Days
UserMatchHistory linkedin.com This cookie is used to track visitors so that more relevant ads can be presented based on the visitor’s preferences. 60,345 Days
bcookie linkedin.com This is a Microsoft MSN 1st party cookie for sharing the content of the website via social media. 2 years
__asc ltimindtree.com This cookie is used to collect information on consumer behavior, which is sent to Alexa Analytics. 1 Day
__auc ltimindtree.com This cookie is used to collect information on consumer behavior, which is sent to Alexa Analytics. 1 Year
_gcl_au ltimindtree.com Used by Google AdSense for experimenting with advertisement efficiency across websites using their services. 3 Months
bscookie linkedin.com Used by the social networking service, LinkedIn, for tracking the use of embedded services. 2 years
tempToken app.mirabelsmarketingmanager.com When the browsing session ends
ELOQUA eloqua.com Registers a unique ID that identifies the user’s device upon return visits. Used for auto -populating forms and to validate if a certain contact is registered to an email group . 2 Years
ELQSTATUS eloqua.com Used to auto -populate forms and validate if a given contact has subscribed to an email group. The cookies only set if the user allows tracking . 2 Years
IDE doubleclick.net Used by Google Double Click to register and report the website user’s actions after viewing clicking one of the advertiser’s ads with the purpose of measuring the efficiency of an ad and to present targeted ads to the user. 1 Year
NID google.com Registers a unique ID that identifies a returning user’s device. The ID is used for targeted ads. 6 Months
PREF youtube.com Registers a unique ID that is used by Google to keep statistics of how the visitor uses YouTube videos across different web sites. 8 months
test_cookie doubleclick.net This cookie is set by DoubleClick (which is owned by Google) to determine if the website visitor’s browser supports cookies. 1,073,201 Days
UserMatchHistory linkedin.com Used to track visitors on multiple websites, in order to present relevant advertisement based on the visitor’s preferences. 29 days
VISITOR_INFO1_LIVE youtube.com 179 days
.avia-table-4 td:nth-of-type(1):before { content: 'Cookie Name'; } .avia-table-4 td:nth-of-type(2):before { content: 'Domain / Associated Domain / Third-Party Service'; } .avia-table-4 td:nth-of-type(3):before { content: 'Description'; } .avia-table-4 td:nth-of-type(4):before { content: 'Retention period'; }
Third party companies Purpose Applicable Privacy/Cookie Policy Link
Alexa Show targeted, relevant advertisements https://www.oracle.com/legal/privacy/marketing-cloud-data-cloud-privacy-policy.html To opt out: http://www.bluekai.com/consumers.php#optout
Eloqua Personalized email based interactions https://www.oracle.com/legal/privacy/marketing-cloud-data-cloud-privacy-policy.html To opt out: https://www.oracle.com/marketingcloud/opt-status.html
CrazyEgg CrazyEgg provides visualization of visits to website. https://help.crazyegg.com/article/165-crazy-eggs-gdpr-readiness Opt Out: DAA: https://www.crazyegg.com/opt-out
DemandBase Show targeted, relevant advertisements https://www.demandbase.com/privacy-policy/ Opt out: DAA: http://www.aboutads.info/choices/
LinkedIn Show targeted, relevant advertisements and re-targeted advertisements to visitors of LTI websites https://www.linkedin.com/legal/privacy-policy Opt-out: https://www.linkedin.com/help/linkedin/answer/62931/manage-advertising-preferences
Google Show targeted, relevant advertisements and re-targeted advertisements to visitors of LTI websites https://policies.google.com/privacy Opt Out: https://adssettings.google.com/ NAI: http://optout.networkadvertising.org/ DAA: http://optout.aboutads.info/
Facebook Show targeted, relevant advertisements https://www.facebook.com/privacy/explanation Opt Out: https://www.facebook.com/help/568137493302217
Youtube Show targeted, relevant advertisements. Show embedded videos on LTI websites https://policies.google.com/privacy Opt Out: https://adssettings.google.com/ NAI: http://optout.networkadvertising.org/ DAA: http://optout.aboutads.info/
Twitter Show targeted, relevant advertisements and re-targeted advertisements to visitors of LTI websites https://twitter.com/en/privacy Opt out: https://twitter.com/personalization DAA: http://optout.aboutads.info/
. .avia-table tr {} .avia-table th, .flex_column .avia-table td { color: #343434; padding: 5px !important; border: 1px solid #ddd !important; } .avia-table th {background-color: #addeec;} .avia-table tr:nth-child(odd) td {background-color: #f1f1f1;}
Save settings
Cookies settings