Natural Language processing to communicate successfully in human language • Summarizing details to insights • Generating charters • Creating data, instructions and oracles • Understanding risk coverage 02 Knowledge representation to store what it knows or hears • Remembering features and recognizing feature relationships • Avoiding reporting accepted problems without change in knowledge • Remembering what was done • Knowing what to look at 03 Automated reasoning to answer questions and draw new conclusions • Deciding what conversations to start • Deciding when we can automatically revert • Reporting with repro scripts • Recognizing responsibility of fix 04 Machine learning to adapt to new circumstances and to extrapolate and detect patterns • Bug taxonomies • Priorities • Cross industry reuse of standard tests 05 Computer vision, speech recognition to perceive the world • Multilingual projects • Sources of data • Visual testing aids 06 Robotics to manipulate objects and move around • Robotic process automation as basis of testing • Operating interfaces abstracting away technology of target Any software is marketed as AI since it is doing something humans could do.
Text Text /code Sound Image Video Copyrights Training Data Machine Learning Algorithm e.g., LLM input Filter output Filter Fore- ground Back- ground Retrieval pipeline Text Text /code Sound Image Video Copyrights Parameters e.g., temperature Tasks Prompt engineering is the discipline of formulating future background requests in the foreground Copyrights From 2048 tokens (1500 words) to 1M
Down 6 Shift Left Shift down Quality engineering over quality assurance Whole team approach Clarifying core examples Single-commit delivery Test-driven development Exploratory unit testing “77% of production bugs could have been found with a unit test” We’re still half-way into the automation transformation that AI supports.
Test code generation in IDE 1 AI tools in review stage and pipelines in general ’agentification’ 2 Creating a RAG- assistant for testing 3 Applying today’s AI in testing
to corporation trust set up by contracts. Context size Input to general purpose model is of relevant size. Packaging as services Abstractions hiding setup that is not AI on side of AI creating hybrid products. Task-specific application External imagination on level of tasks with human in loop.
Testing 11 Idea of change *Intent* Pull Request Commit Commit Build and Deploy write code write tests document review review codiumai pr-agent pro code review by gpt github action write tests github copilot codiumai codiumate earlyai jetbrains ai assistant run tests run tools run tools IDE nodes Agents running in
that I get with it, it feels like I never have to write any kind of boilerplate code anymore, and I also find it very useful to just ask stuff directly in the IDE. I used to google stuff all the time, and ended up on Stackoverflow a lot, but nowadays I rarely have to do that.” 12
in IDE Chat, inline and in IDE Participants: @workspace Variables: #file #selection #editor Commands: /fix /tests /explain /doc In GitHub Pull request summaries Knowledge bases Spec-Plan-Implement flows Bing for recent ADO for pipelines and environments Lines and blocks *Enterprise Only
argue for different stances on assumptions Recognize insufficiency and fix it – creating average text is not *your* goal Freedom to criticize as the pair takes no offense Dare to ask things you’d not dare to ask from a colleague Co-piloting allows for repair 19 Photo by Rajvir Kaur on Unsplash
be useful in framing 24 Automating leaves behind the most difficult 10%. Debugging your own creation is twice as hard as creating it in the first place. 50% of jobs change, 5% of jobs replaced by automation Software industry doubles in 5 years. 1% (4 min) better daily would make 37.8x you in 365 days. There’s plenty of things we want but can’t get to because creating what we have now hogs all bandwidth.
CGI Finland 01 AI transformation Applying ML, GenAI, RAG, CoT in human-centric contemporary exploratory testing 02 Automation transformation Decomposing to actionable feedback and capabilities in pipelines 03 Testing in the world The Creative Commons baseline and integrations with community 04 Testing in CGI Real work with our customers
Founded in 1976, CGI is among the largest IT and business consulting services firms in the world. We are insights-driven and outcomes-based to help accelerate returns on your investments. Across hundreds of locations worldwide, we provide comprehensive, scalable and sustainable IT and business consulting services that are informed globally and delivered locally. cgi.com
maaretp
kimura_m_29
sumiyae
okuzawats
tera_ny
eiganken
kubotak
lamodatech
ivargrimstad
makihiro
daisukeshinoku
osakatechlab
h0r15h0
notwaldorf
speakerdeck
iamctodd
tammyeverts
phodgson
paigeccino
brad_frost
wjessup
sferik
smashingmag
bkeepers
morganepeng
