akeneo/batch-bundle in a nutshell

Transcript

1. akeneo/batch-bundle in a nutshell Julien Janvier @jujanvier
2. Tool to be able to process large volume of data We want to a “read” large amount of data, process/compute them, and “write” the result somewhere. • But PHP is not made for that oO… • Do not duplicate our business code :) “ 2
3. Job examples We want to daily import 100K products from a CSV file. We want to export our products via a SOAP webservice. We want to transform a large CSV file to a JSON file. “ “ “ 3
4. Job instances code label job_name type raw_parameters code label job_name type raw_parameters csv_product_import_erp CSV product import ERP csv_product_import import {“filePath”:”\/tmp\/products_erp.csv”,”delimiter”,”;”} csv_product_import_gibson CSV product import Gibson csv_product_import import {“filePath”:”\/tmp\/products_gibson.csv”,”delimiter”:”;”,”enclosure”:”"”} csv_product_export CSV product export csv_product_export export {“filePath”:”\/tmp\/products.csv”} edit_common_attributes Mass edit common product attributes edit_common_attributes mass_edit {} • create a new job instance p h p a p p / c o n s o l e a k e n e o : b a t c h : c r e a t e - j o b . . . 4
5. Job execution • trace each execution of the job • various information: start time, end time, status… 5
6. Job parameters • configure how the job is launched • possible to create a custom class with constraints and default values • launch a job with with a custom configuration with p h p a p p / c o n s o l e a k e n e o : b a t c h : j o b c s v _ p r o d u c t _ i m p o r t _ g i b s o n - - c o n f i g " { \ " f i l e P a t h \ " : \ " / a n o t h e r / p a t h / t o / p r o d u c t . c s v \ " } " 6
7. Steps • where your business code lives • each job has 1 to n steps • implement A k e n e o \ C o m p o n e n t \ B a t c h \ S t e p \ S t e p I n t e r f a c e • possible to pass information between steps • a particular step A k e n e o \ C o m p o n e n t \ B a t c h \ S t e p \ I t e m S t e p • reader • processor • writer 7
8. What to do to use it? 1. implement A k e n e o \ C o m p o n e n t \ B a t c h \ S t e p \ S t e p I n t e r f a c e 2. define the job as a tagged service a k e n e o _ b a t c h . j o b 3. [define a class for the configuration] 8
9. i n t e r f a c e S t e p I n t e r f a c e { p u b l i c f u n c t i o n g e t N a m e ( ) ; / * * * P r o c e s s t h e s t e p a n d a s s i g n p r o g r e s s a n d s t a t u s m e t a i n f o r m a t i o n t o t h e * S t e p E x e c u t i o n p r o v i d e d . T h e S t e p i s r e s p o n s i b l e f o r s e t t i n g t h e m e t a * i n f o r m a t i o n a n d a l s o s a v i n g i t i f r e q u i r e d b y t h e i m p l e m e n t a t i o n . * * @ p a r a m S t e p E x e c u t i o n $ s t e p E x e c u t i o n a n e n t i t y r e p r e s e n t i n g t h e s t e p t o b e e x * * @ t h r o w s J o b I n t e r r u p t e d E x c e p t i o n i f t h e s t e p i s i n t e r r u p t e d e x t e r n a l l y * / p u b l i c f u n c t i o n e x e c u t e ( S t e p E x e c u t i o n $ s t e p E x e c u t i o n ) ; 0 1 . 0 2 . 0 3 . 0 4 . 0 5 . 0 6 . 0 7 . 0 8 . 0 9 . 1 0 . 1 1 . 1 2 . 1 3 . 1 4 . 9
10. a c m e . j o b . c s v _ p r o d u c t _ i m p o r t : c l a s s : ' A k e n e o \ C o m p o n e n t \ B a t c h \ J o b \ J o b ' a r g u m e n t s : - ' c s v _ p r o d u c t _ i m p o r t ' - ' @ e v e n t _ d i s p a t c h e r ' - ' @ a k e n e o _ b a t c h . j o b _ r e p o s i t o r y ' - - ' @ a c m e . j o b . s t e p . f i r s t ' - ' @ a c m e . j o b . s t e p . s e c o n d ' t a g s : - { n a m e : a k e n e o _ b a t c h . j o b } 0 1 . 0 2 . 0 3 . 0 4 . 0 5 . 0 6 . 0 7 . 0 8 . 0 9 . 1 0 . 1 1 . 10
11. Misc • possible to send a mail when the job is finished • events are dispatched • invalid items are handled • opensource - licence OSL 3.0 • installation via c o m p o s e r r e q u i r e a k e n e o / b a t c h - b u n d l e • documentation at docs.akeneo.com 11
12. Thank you :) Questions? Julien Janvier @jujanvier