The New Hybrid Future of Software Engineering

Keynote at ESEM 2022 in Helsinki, Finland.

The COVID-19 pandemic shook the world to its core and provoked an overnight exodus of software developers who normally worked in an office setting to working from home. In this talk, I will present the results of several studies at Microsoft on how software developers and their teams have been affected by this change. I will discuss benefits and challenges of remote work and the impact of productivity during the pandemic. I will conclude with research opportunities and implications for the new hybrid work in software engineering as many developers are returning to work in offices.

https://conf.researchr.org/home/esem-2022

Transcript

1. The New Hybrid Future of Software Engineering Thomas Zimmermann #ESEM2022
2. 2 «Software is eating the world.» Image: Pixels Movie
3. The SPACE of developer productivity 3 Make higher-quality software Produce software more efficiently Make smarter planning decisions Improve the well-being and satisfaction of developers Facilitate team collaboration and success Transition to a fully remote working world Why developer productivity matters Learning how to define, measure, and predict developer productivity can help organizations:
4. The SPACE of developer productivity 4 Productivity is all about developer activity You can’t improve what you don’t understand. Here are the common myths and misconceptions around developer productivity: Myths and misconceptions about developer productivity 01 Productivity is only about individual performance 02 One productivity metric can tell you everything 03 Productivity measures are only useful for managers 04 Productivity comes down to engineering systems and developer tools 05
5. The SPACE of developer productivity 5 SPACE: A framework for understanding developer productivity The SPACE framework offers a new way of thinking about developer productivity. https://aka.ms/SpaceFramework ACMQUEUE – March 2021
6. The SPACE of developer productivity 6 SPACE: A framework for understanding developer productivity The SPACE framework offers a new way of thinking about developer productivity. https://aka.ms/SpaceFramework Satisfaction and well-being Performance Activity Communication and collaboration Efficiency and flow
7. The SPACE of developer productivity 7 SPACE framework in action Individual One person Team or group People that work together Level System End-to-end work through a system (like a development pipeline) Satisfaction & Well-being How fulfilled, happy, and healthy one it Developer satisfaction Retention* Satisfaction with code reviews assigned Perception of code reviews Developer satisfaction Retention* Satisfaction with engineering system (e.g., CI/CD pipeline) Performance An outcome of a process Code review velocity Code review velocity Story points shipped* Code review velocity Code review (acceptance rate) Customer satisfaction Reliability (uptime) Activity The count of actions or outputs Number of code reviews completed Coding time # commits Lines of code* # story points completed* Frequency of deployments Communication & collaboration How people talk and work together Code review score (quality or thoughtfulness) PR merge times Quality of meetings* Knowledge sharing, discoverability (quality of documentation) Efficiency & flow Doing work with minimal delays or interruptions Code review timing Productivity perception Lack of interruptions PR merge times Quality of meetings* Knowledge sharing, discoverability (quality of documentation) Code review timing Handoffs Knowledge sharing, discoverability (quality of documentation) Code review timing Velocity/flow through the system * Use these metrics with (even more) caution – they can proxy more things.
8. Impact of work from home on software engineers at Microsoft (*) (*) during the pandemic
14. Work from Home Denae Ford, Margaret-Anne D. Storey, Thomas Zimmermann, Christian Bird, Sonia Jaffe, Chandra Shekhar Maddila, Jenna L. Butler, Brian Houck, Nachiappan Nagappan: A Tale of Two Cities: Software Developers Working from Home During the COVID-19 Pandemic. ACM TOSEM (2021) https://arxiv.org/abs/2008.11147
15. Multi-survey study How has productivity changed? Why? What is good about working from home? What is bad about working from home? What challenges have you encountered working from home? How has communication with your team members changed since working from home? How do you stay socially engaged with your colleagues (in place of hallway conversation)?
16. Multi-survey study How has productivity changed? Why? What is good about working from home? What is bad about working from home? What challenges have you encountered working from home? How has communication with your team members changed since working from home? How do you stay socially engaged with your colleagues (in place of hallway conversation)?
17. Compared to working in office, how has your productivity changed? (N=2,254, April/May) 11.18% 25.60% 31.72% 25.51% 5.99% Significantly more productive More productive About the same Less productive Significantly less productive Answer Percent Significantly less productive 6.06% Less productive 31.79% About the same 31.79% More productive 22.06% Significantly more productive 8.30% Previous survey: March 16-23, N=1,357
18. What is good about working from home? Workplace – In the current WFH environment respondents reported advantages including now having a window office, no longer being in an open-office space, and feeling safer. Well Being – Quality of a healthy lifestyle has increased (e.g., enabling ease of physical fitness and more healthy diets), they are taking more breaks, as well as reducing health risks from the crisis. Beyond Work – Respondents reported feeling as though they enabled at better work-life balance by having an increased proximity to family, enabled access to higher quality food, and the ability to manage home needs. Flexibility – Respondents described having more control over their tasks and schedule which in turn made them feel more productive. Efficient Interactions – With the increased uncertainty of daily needs, respondents have found it helpful to reduce spontaneous meetings, interruptions, and distractions. Convenience & Commute – Over half of respondents mentioned the reduced commute as a great benefit of now WFH. Other participants in this cluster described the convenient clothing of PJs that have made them feel more comfortable during times of work.
19. How frequent (Prevalence) and how important (Importance) are these benefits? How has productivity changed (Delta) when a benefit is present? Statistically significant differences indicated with (*) What work-related benefits have you experienced working from home?
20. Lasso regression analysis. The dependent variable is whether a participant reported that productivity stayed the same or increased. The explanatory variables are the direct effects and interactions for major benefits experiences and reported to be important.
21. What is bad about working from home? / What challenges have you encountered working from home? Communication and Coordination – Friction interacting with colleagues when working from home. This includes not being able to just drop in on an office, not being able to find them on teams, or not being able to have a rich in person conversation. Lack of ad-hoc meetings and social interactions. Respondents missed the Hallway and other ad-hoc impromptu conversations. Various meeting issues. Respondents mentioned challenges with having online meetings (people speaking over each other, not being able to "read" people when no video is in use, and not being able to brainstorm as easily as when in the same physical room.) Workplace Setup – The work environment at home is worse than the office environment. This includes no specific area to work (e.g., having to sit at a kitchen table or on a couch), temperature, noise, light, poor chair, etc. In some cases, the environment does not provide same ergonomic affordances as the work office (pain from poor chairs or uncomfortable work positions). Many people complained that they had to work on just their laptop, and they missed the power of their desktop setup or having multiple monitors. Hard to Focus, Distractions, and Interruptions – It is harder to maintain focus when working from home. Non-family distractions around the house (laundry, TV, Facebook, etc.) Distractions specifically related to family members or other people living in the same residence. Distractions that arise from work such as emails, Teams chats, etc.
22. What is bad about working from home? / What challenges have you encountered working from home? Connectivity – Working from home is hindered by poor Internet connection, SAW and VPN drops, and having to use remote desktops. Reboots/updates sometimes required employees drive to get machines working again. Work-life balance – Not being able to establish a clear boundary being work life and home life. This includes feeling "always on", putting in more hours for work so that there is less "balance", or switching back and forth between work and home responsibilities frequently. Lack of routine – Working different hours than normal, not having a consistent schedule. Staying in the same place for too long, forgetting to take a lunch or snack break, not moving between meetings. Missing convenience of food/coffee/drinks on campus. Annoyance of having to make one's own food. Physical and mental health – Feeling lonely or isolated. Feeling pressure to be as productive from home as in office, feeling stress/anxiety about job or current virus situation. Finding it more difficult to stay motivated to get work done. People miss exercise in the form of walking around campus, walking between office, or to the break room when working from home. None – Respondents explicitly mentioned no challenges and/or nothing bad about working from home.
23. How frequent (Prevalence) and how impactful (Impact) are these challenges? How has productivity changed (Delta) when a challenge is present? Statistically significant differences indicated with (*) What challenges have you experienced working from home?
24. Lasso regression analysis. The dependent variable is whether a participant reported that productivity stayed the same or increased. The explanatory variables are the direct effects and interactions for major challenges.
25. What could be improved about working from home? (Choose up to three items.)
26. work autonomy work environment meetings ability to focus work life balance childcare The Yin and Yang of Working from Home
27. Charles Dickens’ “A Tale of Two Cities” begins, “It was the best of times, it was the worst of times.” Adapting Dickens’ line to leading an engineering team during the global pandemic, I’d say “We’re doing very well, we’re barely hanging in there.” – Shane O’Flynn What It’s Like to Lead a 100-Person Engineering Team During the Pandemic (lohika.com)
28. Team Productivity Courtney Miller, Paige Rodeghero, Margaret-Anne D. Storey, Denae Ford, Thomas Zimmermann: "How Was Your Weekend?" Software Development Teams Working From Home During COVID-19. ICSE (2021). https://arxiv.org/abs/2101.05877
29. Team productivity Compared to working in office, how has the productivity of your team changed? 3.37% 19.58% 53.77% 21.51% 1.77% Significantly more productive More productive About the same Less productive Significantly less productive 70% say that the ability of the team to meet milestones has not changed. However, there are concerns that this may not be sustainable. As a team we are taking longer to meet milestones mostly due to people's availability changing with having to take care of family, mental health issues, or simply burning out from blurred work/home boundaries. I think that for certain roles which rely on in-person collaboration and also need to align people it's harder and takes more time. So even if the milestones are achieved, the workload increase in effort makes this unsustainable over time. We need to slow down and learn how to get as efficient and smart on doing 100% remote work. My feeling is that our reliance on email and individual contributors instead of teams have delayed our timeliness to resolve issues, and a tool like DrWhom is not enough to identify knowledgeable alternate points of contact.
31. Compared to working in office, do you think the ability of the team to reach milestones has changed? We have lost somewhere between 20%-40% effectiveness in use of time. In order to keep up, people are working longer hours. We are starting to see burnout. (S1384) It is more difficult to land nuanced discussions. 1:1 is OK, but in a multi-person discussion it is very, very hard to keep focus. (S1777) The time pressure due to child care expectations and more-frequent scheduled meetings (vs. quick hallway chats) has definitely slowed the team down. We have explicitly and officially postponed some work, citing Coronavirus-rooted challenges as reasons. (S887) Brainstorming activities that need whiteboard and involve more passionate conversation progresses slower than before and can be frustrating. (S1833) Trouble with collaboration and communication
32. How has your team culture changed? We used to have lunches together, and had good comeraderie. No more lunches together, less comeradie, and less incidental ‘water cooler’ information exchange. (S2049) We are trying to adapt to daily sync ups and face time between all team members. which helps with communication but adds into the million meetings. (S1722) More explicit efforts to facilitate social events and foster team relationships. (S1676) I feel no difference. (S1801) Change in Social Connections Change in Communication
33. How has your team supported you during this crisis? 35 We have weekly social hours, which helps with some of the social isolation I’ve been feeling. (S1562) Everyone is supportive of each other. It is easy to talk with someone if needed. Feel supported. (S1053) Lots of teams meetings. (S169) More meetings with our direct manager’s team just to talk and check in. (S1636) Focus on Social Connections Maintaining Communication
34. How has communication with your team members changed since working from home? Frequency: Much less communication for most, especially less informal/social communication, more meetings for some Tools: Heavy use of Teams, more video, more emails (some less), some miss whiteboards, more miss F2F interactions Challenges: Communication tool friction, feeling isolated (while a few mention they feel more connected) Adapting: Emerging new practices such as daily standups, specific Teams channels for topics/social interactions, and more intentional communication (can’t rely on drivebys) Effectiveness: Varies between same, worse, better "Less frequent. More intentional. You don't overhear as many team conversations unless those happen to be posted in a Teams channel. Sometimes there is an overload of too many communication channels: email, multiple Teams' teams, channels, chat groups, Slack for some external partnerships, GitHub issue discussions, Azure DevOps issue discussions
35. How has communication with your team changed since working from home? 37
36. AIC Backwards Stepwise Regression Team culture factor Model with All Factors Team culture factor Team culture factor Team culture factor Final Reduced Model
37. Explaining Team Productivity Ability to make decisions as a team Impactful contributions I make to team Ability to brainstorm with team External team bonding activities Difficulty communicating with colleagues Less awareness of what colleagues are working on Start/end meetings with non-work talk Frequency of scheduled social activities Ability to brainstorm with team Team members are Supportive of new ideas
38. Recommendations for effective remote teams Build and maintain team culture. Include social activities as part of “work.” Be mindful of other people’s time. Actively work to be inclusive. Managers should stay aware of contributions. Managers should hold weekly 1:1 meetings. Hold regular team meetings. Create communication standards.
39. Onboarding Paige Rodeghero, Thomas Zimmermann, Brian Houck, Denae Ford: Please Turn Your Cameras On: Remote Onboarding of Software Developers during a Pandemic. ICSE SEIP (2021) https://arxiv.org/abs/2011.08130
40. Methodology Sent a detailed survey to FTEs who have recently onboarded Performed open coding on the data collected for a qualitative analysis Conducted a quantitative analysis on the closed question responses Conducted interviews with new hires and early- career employees 43
41. Survey Participant Overview • 1000 emails sent for recruitment • MSFT FTE developers • North America • Start date had to be in 2020 • 267 responses collected (26.7% response rate) • All participants were working at home at the time of the survey • Gender of participants • 57 women • 162 man • 1 non-binary/ gender diverse 44
42. RQ1: Pandemic Onboarding Challenges What challenges have you faced while onboarding to your team and project? Communication and Collaboration Asking for Help Building Team Connections Finding Documentation Technical Issues Hardware and Permissions “Because everyone is working from home I don’t run into people randomly or walk by their office, which has led me to ask for less help than I usually would have because I don’t like asking small, half-thought out questions virtually.” (S106) “During meetings, most teammates keep the camera off. This makes collaboration and remote experience difficult.” (S228) “Onboarding remotely is definitely more challenging. The inability to truly interact with coworkers, the in-ability to really have traditional team building, and the disconnection from everyone contributes to feeling like my work is kind of isolated from everyone else. It also makes it harder to get a broader, high level understanding of the team and organization.” (S91) 45
43. RQ2: Team Onboarding Efforts What has the team done to make you feel connected to the team? Frequent 1:1 meetings with manager Remote team meeting 1st day Checking in to prevent blockages Social virtual events Onboarding buddy Assigned mentor “The team has been extremely welcoming. My very first day we met over teams and each person introduced themselves and welcomed me personally. Several people have reached out to ask how my onboarding is going and offered to answer any questions I have. Several members of my team invited me to join them in a hackathon. Generally everyone has been incredibly helpful and patient when teaching me.” (S191) 46
44. RQ3: Team Interaction with New Hires 129 (58.1%) new hires reported 1:1 meetings daily Communication focused on work related topics 72.9% of the communication lasted < 30 minutes 43.4% of the communication was < than 15 minutes “Formally there are scheduled meetings - mostly with the whole team, but I do have a scheduled 1:1 with my manager. There are informal chats and some email exchanges between me and individuals to help me.” (S18) 47
45. RQ4: Social Connection to Team 83.3% of the new hires felt connection to their team “Unfortunately I guess I haven’t taken active steps to do so since I feel like I’m spending al l of my time trying to ramp up and learn.” (S92) “I try to reach out to random team members. I’m soon going to ask my manager if people can start turning on their cameras so I can see facial expressions.” (S75) 48
46. Recommendations for remote onboarding Promote communication & asking for help. Encourage teams to turn cameras on. Schedule frequent 1:1 meetings. Provide information about the organization. Emphasize team building. Assign an onboarding buddy. Assign an onboarding technical mentor. Support multiple onboarding speeds. Assign a simple first task. Provide up-to-date documentation.
47. Other research on the impact of work from home on software engineers (*an incomplete list)
48. “Software companies should focus on maximizing employee wellbeing and improving the ergonomics of employees’ home offices. Women, parents and disabled persons may require extra support.” Empir. Softw. Eng. 25(6): 4927-4961 (2020)
49. “We find that WFH has both positive and negative impacts on developer productivity in terms of different metrics, e.g., the number of builds/commits/code reviews.” Science China Information (2022)
50. “We identified that 74.1% of the participants consider their productivity remains good or excellent, and 84.5% feel motivated and have easy communication with their co-workers. Furthermore, the main factors influencing the productivity are external interruption, environment adaptation, and emotional issues.” SBES (2020)
51. “Overall, we conclude that working from home was per se not a significant challenge for software engineers.” Empir. Softw. Eng. (2021) ICSE SEIP (2021)
52. Key Findings: (1) Developer activity remains largely consistent or increased compared to last year. (2) Flux in GitHub issues likely reflect the shift to distributed work and local virus responses. (3) The cadence of work has changed. (4) Patterns of developer activity have implications for burnout. (5) People are collaborating more, especially on open-source projects.
53. Findings: (1) Behind the average “no change” lays a large variation of experiences. (2) Productive engineers benefit from a better organization of work, increased flexibility and focus. (3) Poor collaboration, emotional issues, distractions and poor equipment hinder productivity. (4) Average productivity in later months is higher than the early months of the pandemic. JSS (2022)
54. New Future of Work Symposium (2020)
55. CHI EA (2021) Identified people’s diverse strategies and reasons behind the decisions of where to work during the hybrid work phase.
56. http://aka.ms/newfutureofwork
57. The New Future of Work Hybrid
58. Everything Everywhere All at Once A 2022 American absurdist science fiction comedy-drama film written and directed by Daniel Kwan and Daniel Scheinert (collectively known as "Daniels"). It stars Michelle Yeoh as a Chinese American woman being audited by the IRS who discovers that she must connect with parallel universe versions of herself to prevent a powerful being from causing the destruction of the multiverse.
59. Enable everyone to work anywhere with anyone at anytime on any device
60. Enable everyone Broaden the participation of developers. How can we leverage AI to allow more people to contribute to software?
61. To work anywhere
62. With anyone Need for physical co-location less important. Teams will be more distributed. Inclusive communities as a tool to facilitate this work. Collaborators can be anywhere as well. At anytime Asynchronous collaboration On any device Desktop, laptop, phone
63. What are research opportunities for ESEM? • Empirical studies of the changing work practices. • Well-being, mental health, and burnout in software engineering. • How to design inclusive communities. • Leverage AI to improve software development productivity throughout the entire development lifecycle • Broaden participation in software development, for example, by leveraging AI and similar techniques.