Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Testing Apps at Facebook

Testing Apps at Facebook

How do Facebook engineers ship features to a billion people every two weeks? Learn how we continuously test mobile applications. We will also talk about stress testing and how to measure the impact of code changes on performance.

Martin Konicek

October 31, 2014
Tweet

More Decks by Martin Konicek

Other Decks in Technology

Transcript

  1. Automated cut every 2 weeks 2 weeks of development master

    release branch 2 weeks of stability 2 weeks of development Release
  2. Unit tests As a first line of defense ! !

    ! ! ! $ buck test javatests/com/facebook/timeline/ PASS 10 Passed 0 Failed TimelineDbCacheTest PASS 16 Passed 0 Failed TimelineRamCacheTest
 PASS 4 Passed 0 Failed FetchTimelineHeaderMethodTest
 ...
 TESTS PASSED

  3. Unit tests As a first line of defense ! !

    ! Mocked: ▪ Backend ▪ The OS ! $ buck test javatests/com/facebook/timeline/ PASS 10 Passed 0 Failed TimelineDbCacheTest PASS 16 Passed 0 Failed TimelineRamCacheTest
 PASS 4 Passed 0 Failed FetchTimelineHeaderMethodTest
 ...
 TESTS PASSED

  4. ! ! 
 public class AudiencePickerTest { ! private DroidDriver

    driver; private Wait<DroidDriver> wait; private NewsFeed newsFeed; ! @Before public void login() { driver = DroidDriver.startApp(Apps.FB4A); wait = new DroidWait(mDriver, Duration.seconds(20)); ! TestUser user = TestUser.create(); newsFeed = new Login(driver).as(user); }
 ! ! !
  5. ! @Test public void changeAudience() { AudiencePicker audiencePicker = newsFeed

    .goToStatusComposer() .openAudiencePicker() .selectAudience(Audience.FRIENDS); .type("Droidcon London is the best!”) .post(); ! // Check the story in the backend FeedStory story = wait.until(GraphApi.visibilityOfStory(user)); assertEquals( "Story audience should respect selection", Audience.FRIENDS, story.getAudience()); }

  6. What we measure ▪ Time to perform an action ▪

    Allocations ▪ FPS ▪ Memory, network, CPU usage ▪ Anything (extensible framework)
  7. Measuring time Challenges
 
 

 The app provides timing information


    
 $ adb logcat | grep CommentsLoad ! D/fb4a:PerformanceLogger: Name: CommentsLoad; Timestamp: 9318686 D/fb4a:PerformanceLogger: Name: CommentsLoad; Timestamp: 9318722; Elapsed: 36
 ...
  8. Measuring time Challenges $ adb logcat | grep CommentsLoad |

    grep Elapsed | awk -F';' '{print $3}'
 ! Elapsed: 46 Elapsed: 66 Elapsed: 76 Elapsed: 36 Elapsed: 40 Elapsed: 83 Elapsed: 34 Elapsed: 36 Elapsed: 33
  9. Measuring time Challenges ▪ Garbage collection ▪ Thread scheduling
 


    ! ! ▪ Network ▪ Overhead of measuring code
 

  10. Measuring time Challenges ▪ Garbage collection ▪ Thread scheduling
 


    ! ! ▪ Network ▪ Overhead of measuring code
 

  11. @RunWith(PerformanceTestRunner.class) @TestRepetitions(50) public class CommentsLoadTest { ! ... ! @BeforeRepetition

    public void loginUser() { ...
 driver.disableNewsFeedAutoRefresh(); newsFeed = new Login(driver).as(user); } ! @Test public void openComments() { Button b = newsFeed.getFirstStory().getFeedbackButton(); ! driver.gc(); ! b.click(); waitForMarkers("CommentsLoad", 5, TimeUnit.SECONDS); } ! @After public void goBackToNewsFeed() { driver.sendKeys(SelendroidKeys.BACK); } }
  12. Garbage Collection in Dalvik $ adb logcat
 
 ...
 dalvikvm(6316):

    GC_CONCURRENT freed 431K, 7% free 12898K/13831K, paused 1ms+2ms dalvikvm(6316): GC_CONCURRENT freed 381K, 7% free 12928K/13831K, paused 1ms+2ms dalvikvm(6316): GC_CONCURRENT freed 433K, 7% free 12936K/13831K, paused 1ms+3ms dalvikvm(6316): GC_CONCURRENT freed 121K, 5% free 13211K/13831K, paused 1ms+2ms ▪ Non-compacting

  13. Garbage Collection in Dalvik $ adb logcat
 
 ...
 dalvikvm(6316):

    GC_CONCURRENT freed 431K, 7% free 12898K/13831K, paused 1ms+2ms dalvikvm(6316): GC_CONCURRENT freed 381K, 7% free 12928K/13831K, paused 1ms+2ms dalvikvm(6316): GC_CONCURRENT freed 433K, 7% free 12936K/13831K, paused 1ms+3ms dalvikvm(6316): GC_CONCURRENT freed 121K, 5% free 13211K/13831K, paused 1ms+2ms ▪ Non-compacting ▪ Non-generational

  14. Garbage Collection in Dalvik $ adb logcat
 
 ...
 dalvikvm(6316):

    GC_CONCURRENT freed 431K, 7% free 12898K/13831K, paused 1ms+2ms dalvikvm(6316): GC_CONCURRENT freed 381K, 7% free 12928K/13831K, paused 1ms+2ms dalvikvm(6316): GC_CONCURRENT freed 433K, 7% free 12936K/13831K, paused 1ms+3ms dalvikvm(6316): GC_CONCURRENT freed 121K, 5% free 13211K/13831K, paused 1ms+2ms
 
 ...
 dalvikvm(6316): GC_FOR_ALLOC freed 762K, 13% free 4116K/4688K, 
 paused 181ms, total 182ms ▪ Non-compacting ▪ Non-generational