Apidays London 2024 - Open Standards, AI and Data for Better Decisions by Ravinder Singh

Challenges and Opportunities in API Utilisation and Governance Across Global Governments: A Study of Best Practices and Success Stories
Dr. Ravinder Singh Zandu, Head of Digital and Systems Team - Cabinet Office - Government Commercial Function (GCF)

apidays London 2024 - APIs for Smarter Platforms and Business Processes
September 18 & 19, 2024

------

Check out our conferences at https://www.apidays.global/

Do you want to sponsor or talk at one of our conferences?
https://apidays.typeform.com/to/ILJeAaV8

Learn more on APIscene, the global media made by the community for the community:
https://www.apiscene.io

Explore the API ecosystem with the API Landscape:
https://apilandscape.apiscene.io/

Transcript

1. Dr. Ravinder Singh Modernising Technology Programme Manager Cabinet Office-Central Digital and Data Office ravinder.singh@digital.cabinet-office.gov.uk
2. Open Standards, AI and Data for better business decisions
3. Source: domo CDDO
4. >2.6 quintillion bytes Data Generated everyday By 2025 : 463 exabytes per day Instagram photos and videos : 95 million Every day, 306.4 billion emails are sent, and 500 million Tweets are made. Year Data Generated Change Over Previous Year 2020* 64.2 zettabytes ↑ 23.2 zettabytes 2021* 79 zettabytes ↑ 14.8 zettabytes 2022* 97 zettabytes ↑ 18 zettabytes 2023* 120 zettabytes ↑ 23 zettabytes CDDO
5. Every minute of the day: • Number of Google Searches: 5.9mn • WhatsApp Messages: 41.7mn • TikTok Videos: 167 mn • $3805 for mobile Apps • On Facebook: 510 K comments are posted, 293 K statuses are updated, and 136 K photos are uploaded. • Instagram photos: 55 K • Twitter Tweets: 0.5 mn • Youtube videos seen: 4.5 mn • Instagram posts: 277 K • Tinder: 1.4 mn Swipes • 188 mn emails • 103 mn spam mails • 18 mn text messages Every Day: • 2.9 bn email users • Number of Facebook users: 2.38 bn • Facebook logon daily: 1.56 bn • Mobile Facebook users: 1.74 bn • Fake profiles: 83 mn • Instagram users: 600 mn (400 mn active every day) • 95 mn photos and videos shared in Instagram • 100 mn use stories on Instagram • 232 K skype calls
6. BI to AI
7. Image: University of Wisconsin
8. Ø Transparency and accountability: AI-ML remains a black box Ø Data Quality and Algorithm Ø Data Availability and integrity Ø Data privacy and security issues. Ø Resource / lack of expertise Ø Workforce development Ø Regulations Ø Legal and ethical considerations Ø Security Ø Legacy systems and Integration CDDO Current challenges
9. Ø Enhanced Efficiency and productivity Ø Improved service delivery to citizens Ø Data-Driven Decision Making Ø Citizen Engagement and Participation Ø Cost savings and resource optimization Ø Enhanced transparency and accountability Ø Policy Writing Ø Writing Business case Ø Writing Project reports Ø Software coding to develop bespoke or customisation of existing services Ø Medical field CDDO Opportunities
10. $5bn to $30bn £1bn China (2017-2030) UK, backed by £300mn of private investment Beijing - $2bn Tanjin $16bn $1.8bn $ 1bn France USA 47% Employment opportunities taken by machines It has been estimated that AI could add an additional USD $814 billion (£630bn) to the UK economy by 2035, increasing the annual growth rate of GVA from 2.5 to 3.9%. Estimated that AI has the potential to add $957 billion to India’s GDP by 2035 and boost India’s annual growth by 1.3 percentage points by 2035. CDDO
11. CDDO Stages of the Machine Learning Process
12. CDDO Continuous ML Model and Control Framework
13. Judgement Based Rules Based Programmed Strictly Controlled Contained Transformational Tactical Business Impact Self-Learning Autonomous Unbounded Automation Platform Foundation Automation VB Scripts, Excel Macros, Hotkeys, Unified Desktop Artificial Intelligence Virtual Assistance Robotics Process Automation CDDO
14. “By 2021, The RPA Market value, While Large, It's Only A Subset Of The $48.5 Billion Broader AI Cubicle Spend”, Forrester 2017 of the world’s leading organizations will implement a Chief Robotics Officer role and/or define a robotics- specific function within the business by 2019. IDC Worldwide Robotics 2017 Predictions-Jing Bing Zhang 30% 3M workers globally will be supervised by a “roboboss” by 2018, GARTNER A world in which your organisation is 15% human and 85% robots? $2.9B of all customer interactions will be handled without a human agent by 2020, GARTNER 85% $9T 1/3 of the workforce may be replaced by robots by the year 2025, GARTNER “Employment costs could be cut with AI- enabled automation of knowledge work by 2020”, Everest, Bank of America
15. CDDO
16. CDDO Generative AI and LLM • A branch of AI/ML that focuses on creating or generating new content, such as images, text, audio, or even entire virtual worlds, based on patterns and data it has been trained on. • Unlike traditional AI systems that primarily focus on recognizing patterns and making predictions based on existing data, generative AI goes a step further by generating new and original content that closely resembles the data it has learned from. • Generative AI techniques typically rely on deep learning models, such as generative adversarial networks (GANs), variational autoencoders (VAEs), or transformers. • Generative AI has various applications across different domains. It can be used for creative purposes, such as generating artwork, music, or storytelling Image: BCV
17. Internet of Things (IoT) CDDO Three Key components • Things/Devices: physical objects or devices equipped with sensors, actuators, and connectivity capabilities. • Connectivity: Wi-Fi, Bluetooth, cellular networks, low-power wide-area networks (LPWAN), or other wireless protocols. • Cloud and Data Processing: The data collected by IoT devices is sent to cloud- based platforms or edge computing systems, where it is processed, analysed, and stored. IoT • Smart Homes • Industrial IoT (IIoT) • Healthcare IoT • Agriculture IoT
18. CDDO
19. Interoperability
20. We support and encourage the use of open standards in government CDDO
21. "Open Standards" are standards made available to the general public and are developed (or approved) and maintained via a collaborative and consensus driven process. "Open Standards" facilitate interoperability and data exchange among different products or services and are intended for widespread adoption. CDDO
22. • Lower Costs, Vendor Neutrality and no Lock-in • No boundaries, more choices and variety • Interoperability and Collaboration • Increases Innovation • No Licence Management • Easier portability of application • Regular Updates and Quick Fixes • Flexibility and easier customisation • Robustness and durability • Increased available skills • Efficient use of existing resources • Higher Quality Software • More options provide more opportunities to optimise Advantages of using Open Standard CDDO
23. CDDO
24. • Digital transformation engagement models • Integration of applications, data and systems • Optimise channel performance while still maintaining business continuity CDDO
25. • Address disparity • Take advantage of expertise of the back- end systems -Increased productivity • Enable innovation with data with more accurate and trustworthy data • Derive enterprise-grade artifacts • Faster decision-making • Cost-effectiveness CDDO
26. • Legacy system integration • Enterprise application integration (EAI) • Third-party system integration • Business-to-business integration
27. Ways to connect systems • Middleware • Webhooks and Message Queues • Electronic Data Interchange (EDI) • Application Programming Interfaces (APIs) CDDO
28. System integration Approaches • Point-to-point integration model (P2P) Star/Spaghetti • Hub-and-spoke model • Enterprise Service Bus (ESB) model • Integration Platform as a Service (iPaaS) CDDO
29. Application Programming Interface (API)
30. Why we need APIs • Improved collaboration: The average enterprise uses almost 1,200 cloud applications • Easier innovation and flexibility: A company began as an API with just seven lines of code. The company has since partnered with many of the biggest enterprises in the world, diversified to offer loans and corporate cards, and was recently valued at USD 36 billion • Data monetization: API economy initially free and then charge. When AccuWeather launched its self-service developer portal to sell a wide range of API packages, it took just 10 months to attract 24,000 developers, selling 11,000 API keys and building a thriving community in the process. • Added security: APIs create an added layer of protection between your data and a server. Developers can further strengthen API security by using tokens, signatures and Transport Layer Security (TLS) encryption; by implementing API gateways to manage and authenticate traffic CDDO
31. How do APIs work CDDO
32. Traditional integration architecture Centrally provisioned and administered Modern integration architecture Fine-grained deployment, decentralized ownership, and cloud native infrastructure CDDO
33. Common API examples • Universal logins • Third-party payment processing • Travel booking comparisons • Google Maps • Twitter CDDO
34. • Open APIs are open source application programming interfaces you can access with the HTTP protocol. Also known as public APIs, they have defined API endpoints and request and response formats. • Partner APIs are application programming interfaces exposed to or by strategic business partners. Typically, developers can access these APIs in self-service mode through a public API developer portal. Still, they will need to complete an onboarding process and get login credentials to access partner APIs. • Internal APIs are application programming interfaces that remain hidden from external users. These private APIs aren't available for users outside of the company and are instead intended to improve productivity and communication across different internal development teams. • Composite APIs combine multiple data or service APIs. These services allow developers to access several endpoints in a single call. Composite APIs are useful in microservices architecture where performing a single task may require information from several sources. Types of APIs CDDO
35. Types of API Protocol • SOAP (Simple Object Access Protocol) is an API protocol built with XML, enabling users to send and receive data through SMTP and HTTP. With SOAP APIs, it is easier to share information between apps or software components that are running in different environments or written in different languages. • XML-RPC is a protocol that relies on a specific format of XML to transfer data, whereas SOAP uses a proprietary XML format. XML-RPC is older than SOAP, but much simpler, and relatively lightweight in that it uses minimum bandwidth. • JSON-RPC is a protocol similar to XML-RPC, as they are both remote procedure calls (RPCs), but this one uses JSON instead of XML format to transfer data. Both protocols are simple. While calls may contain multiple parameters, they only expect one result. • REST (REpresentational State Transfer) is a set of web API architecture principles, which means there are no official standards (unlike those with a protocol). To be a REST API (also known as a RESTful API), the interface must adhere to certain architectural constraints. It’s possible to build RESTful APIs with SOAP protocols, but the two standards are usually viewed as competing specifications. CDDO
36. Microservice Architecture A microservices architecture consists of a collection of small, autonomous services. Each service is self- contained and should implement a single business capability within a bounded context CDDO
37. Big data architecture A big data architecture is designed to handle the ingestion, processing, and analysis of data that is too large or complex for traditional database systems
38. IoT architecture Internet of Things (IoT) is a specialized subset of big data solutions. The following diagram shows a possible logical architecture for IoT. The diagram emphasizes the event-streaming components of the architecture CDDO
39. API Standard Ten Commandments • Have you considered whether the need for your API endpoint is most appropriately served by a new API or an existing API (or extending existing functionality)? • Have you got confidence that an external developer with no additional context can understand and integrate with your API? • Have you defined appropriate Authorisation requirements for your API? • Have you considered the risks and potential impact associated with exposing your API? • Have you established owners for the key processes required for your API? • Have you built a mature understanding of volumetrics (or expected volumetrics) of your API and what requirements you have to meet them? • Have you comprehensively defined the performance metrics that your API needs to capture? • Have you defined Transaction Monitoring / Auditing events that are appropriate for your API? • Have you considered support model requirements for your API? • Have you ensured that your API has been built with the expected level of technical quality? CDDO
40. • ONS - Integrated Data Services • UKRI – Administrative Data Research • Amazon uses Big Data, and the warehouse closest to the customer is selected and shipping costs to be considerably reduced by 10-40% • 1.71 bn Facebook users- targeted marketing and influences • Election result influence • Open Data • Open API CDDO
41. CDDO
42. CDDO
43. CDDO
44. Thanks.. ravinder.singh@digital.cabinet-office.gov.uk