25.5 C
Casper
Friday, July 26, 2024

Data Management is Becoming More Important and Challenging

Must read

Khushbu Raval
Khushbu Raval
Khushbu is a Senior Correspondent and a content strategist with a special foray into DataTech and MarTech. She has been a keen researcher in the tech domain and is responsible for strategizing the social media scripts to optimize the collateral creation process.

Bill Waid, General Manager of Decision Management at FICO, discusses data management’s escalating importance and complexities in the digital transformation era.

“More organizations are turning to data disciplines maturing around and directly related to AI and ML to help manage, optimize, and analyze increasing data flows,” says Bill Waid, General Manager of Decision Management FICO. In this interview, Waid talks about why organizations need to bring in data, map it, and extract value out of it for successful digital transformation and how alternative data enhances the accuracy of consumer credit profiles

Excerpts from the interview:

Is data management still a fundamental challenge? Why?

Data management is becoming more important and challenging. Ninety percent of the world’s data was created in the last two years, with no signs of slowing down. As more and more organizations embark on their digital transformation journeys, they will need to have the ability to bring in data and effectively move it, map it, operationalize it, and extract value from it. 

Organizations must provide a holistic view of internal and external data and end-to-end data integration, transformation, and management tools to do this. In addition to streamlining business processes and reducing waste, AI solutions will create more data and inform customized digital experiences, relevant offers, and features/products that customers want. 

We’re seeing more organizations turn to data disciplines that are maturing around and directly related to AI and ML to help manage, optimize, and analyze increasing data flows.

How does alternative data enhance the accuracy of consumer credit profiles?

Over the last few years, the field has expanded regarding using alternative data in credit risk assessments. Using alternative data is driven by competitive forces and a growing imperative for a more inclusive economy. Effective alternative data can give a B2C organization an edge over its peers. Assessing this value and comparing it to the cost of data acquisition is a maturing discipline in many organizations. Still, factoring into customer-centric data strategies is becoming an increasingly important trade-off.

An estimated 3 billion adults worldwide don’t have credit and credit records. Responsibly engaging that market in financial services is a priority for many lenders.

However, there are now a variety of sources of alternative data that lenders can leverage to inform credit decisions of potential customers, including:

  • Transaction Data (sourced internally)
  • Telecom / Utility / Rental Data
  • Social Profile Data
  • Clickstream Data
  • Audio and Text Data
  • Social Network Analysis
  • Survey / Questionnaire Data

Based on traditional data, these data sources can add predictive value on margin to credit risk models and demonstrate a consumer’s ability to manage their finances and credit repayment trends.

What are the current technological trends and challenges in the analytics sector?

Incorporating analytics into smart applications and business processes is exploding. It’s enabling unprecedented automation in the back, mid, and front office. This includes AI, ML, and all the techniques and innovations arising around a massive global investment, but we’re still on the frontier. R&D is advancing faster than practitioners can keep up, and the nature of the innovation remains out of mainstream reach, particularly in intensive regulatory environments.

There aren’t a lot of organizations that have integrated a sufficiently diverse architecture for analytic development and execution into end-to-end digital intelligence. For a business that runs from data management and ingestion through data transformation and feature engineering into AI, ML, and traditional analytic models and then into the decision with integrated orchestration and queue management for human or virtual agent intervention along the way. 

Few have addressed business KPI monitoring, and ML-optimized instrumentation can wrap these workloads into composable, semi-autonomous business services. That’s a challenging problem to solve, and while some are on various stages of the journey, we have a lot of work to do.

Organizations’ major challenges are transparency to full lineage, explainability, and the intricate dependency tree. The level of interactive complexity that needs to be managed in the design-time environments is many times more dynamic than in the operational environment. As a case in point, I spoke to a Chief Risk Officer who knows he needs to get his pricing models into compliance to avoid a hefty fine. Still, the dependency impacts inside his operating environment are so opaque that he has decided to pay the fine to buy his organization the time it needs to unwind and map the dependencies that he needs to have confidence in approving a pricing change that won’t cost his business more than the fine. In all our enthusiasm over advances in AI and ML, we sometimes lose sight of the real challenges we’re facing in the market every day to make it all come together.

FICO is focused on regulatory compliance in applying AI and ML. The ethical, safety, and fairness concerns associated with a lack of caution create enormous legal vulnerabilities, business risks, and societal consequences. Senior leadership and boards must understand and enforce immutable AI model governance to drive the responsible use of AI in organizations. They need to establish governance frameworks to monitor AI models to ensure the decisions they produce are accountable, fair, transparent, and responsible.

Business scenario simulation can help us with these types of complex problems. They give the power to safely hypothesize and test “what if” scenarios and use the resulting insights to experiment with business pain points and pivot strategies to achieve higher performance. You can see past the day-to-day minutiae and visualize the long-term impacts of important upstream and downstream changes.

An applied intelligence approach enables organizations to gain analytic insights and operationalize these insights in operational decisions to create business outcomes. More organizations adopt an applied intelligence platform, enabling business users to lead innovation and collaborate more effectively with data science. IT to optimize customer journeys with AI-powered intelligence. Doing so allows organizations to gain and operationalize a richly contextualized view of the customer, apply the appropriate AI and advanced analytic techniques to gain competitive insights, use these findings to make better business decisions, and then put these decisions into action.

How are enterprises using a data mesh approach? Is the trend gaining traction?

Data has always been a highly contextual asset. As the data landscape expands and features become critical resources for our businesses, the trend to deeper contextualization is unavoidable. To the extent that data mesh architectures help us draw lineage between knowledge experts and the deepening context of data, it is a natural fit and will continue to gain relevance. The broader trend is how organizations flex hard to become more data-driven and data-centric. As that happens, we need to virtualize data across the silos in organizations, and data mesh concepts help that a great deal, but many ways of looking at it are equally useful. We must stay focused on solving the problem and be careful to over-rotate on a particular technique.

As data proliferate at ever-increasing velocity, more organizations will turn to strategies like data mesh and platform approaches to improve data management and governance holistically across the business. Additionally, the self-service approach data mesh offers will further empower business users to capture, analyze, and manage data in a tailored fashion. 

How do you leverage AI, ML, automation, and other emerging technologies in your operations?

AI has been used extensively within FICO for more than 30 years. Our fraud modeling group pioneered much of our work in the AI space, obtaining some of the earliest patents on neural nets to detect fraudulent credit card transactions. This happened because the fraud use case is very well suited to AI, which drives highly predictive models with low false-positive rates even in a significant “class imbalance” between the fraud/non-fraud outcomes being predicted. 

FICO’s core business is helping organizations assess risk and then make risk/reward-based decisions informed by advanced analytics. FICO has been developing and operationalizing AI and ML for decades, developing many tools and methods that make AI effective in consumer and small business credit decisions and fraud detection from transaction-oriented (payments) and customer/account-oriented (identity theft, account takeover) perspectives. 

What data science and analytics podcasts do you listen to?

I stay up to speed through reading more than listening. I subscribe to The Sequence to some relevant subscriptions from Medium, and I find McKinsey’s emails useful to stay connected with trends in business. They provide a lot of feedback on industrial applications of AI and ML. I also occasionally find the room to plug into industry seminars about Data and Model Ops, where I find that iron sharpens iron.

More articles

Latest posts