Upgrade to Pro — share decks privately, control downloads, hide ads and more …

AWS Lambda를 통한 Tensorflow 및 Keras 기반...

Beomi
April 18, 2018

AWS Lambda를 통한 Tensorflow 및 Keras 기반 추론 모델 서비스하기

AWS Summit 2018 Seoul 에서 진행한
[AWS Lambda를 통한 Tensorflow 및 Keras 기반 추론 모델 서비스하기]
세션 발표자료

Beomi

April 18, 2018
Tweet

More Decks by Beomi

Other Decks in Programming

Transcript

  1. © 2018, Amazon Web Services, Inc. or Its Affiliates. All

    rights reserved. ੉ળߧ CFPNJ!OFYPODPLS /&90/,PSFB "84-BNCEBܳాೠ 5FOTPSGMPX,FSBTӝ߈୶ۿݽ؛ࢲ࠺झೞӝ
  2. © 2018, Amazon Web Services, Inc. or Its Affiliates. All

    rights reserved. ੷ח ੉ળߧCFPNJ!OFYPODPLS *OUFMMJHFODF-BCT "OUJBCVTJOH5FBN ؘ੉ఠ࠙ࢳ ਢ ূ૑פয݂ XJUI1:5)0/
  3. © 2018, Amazon Web Services, Inc. or Its Affiliates. All

    rights reserved. য়ט੉ঠӝೡղਊٜ ؀ࣁח٩۞׬,FSBT۽ࡅܰѱ٩۞׬ݽ؛ٜ݅੗ ,FSBT9DFQUJPOݽ؛ӝ߈$MBTTJGJDBUJPOݽ؛ٜ݅ӝ %PDLFS "NB[PO-JOVY-BNCEB1BDLBHF;*1 "84-BNCEB 4 "1*(BUFXBZਢࢲ࠺झ 2O" %FFQ-FBSOJOH!/&90/
  4. © 2018, Amazon Web Services, Inc. or Its Affiliates. All

    rights reserved. য়ט੄ҙ੼ "84࠺ਊѐߊ੗࠙ࢳо੄दр࠺ਊ औѱ঳ח࠙࢑୊ܻ ੺؀੸ੋ੉੼ࠁ׮ח࢚؀੸੉੼
  5. © 2018, Amazon Web Services, Inc. or Its Affiliates. All

    rights reserved. ؀ࣁח٩۞׬ 
 ,FSBT۽ࡅܰѱ٩۞׬ݽ؛ٜ݅੗
  6. © 2018, Amazon Web Services, Inc. or Its Affiliates. All

    rights reserved. חঌѷחؘ  ,FSBTחޥоਃ
  7. ,FSBT۽ࡅܰѱݽ؛ٜ݅ӝ • 4FRVFOUJBMݽ؛ୡ݅ীࢎਊೞӝ 4FRVFOUJBM.PEFMࢤࢿ model = Sequential() model.add(Dense(1, activation='sigmoid', input_dim=100))

    from tensorflow.python.keras.models import Sequential from tensorflow.python.keras.layers import Dense import numpy as np 4FRVFOUJBM.PEFM*NQPSU .PEFMDPNQJMF model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy']) 4BNQMFEBUB data = np.random.random((1000, 100)) labels = np.random.randint(2, size=(1000, 1)) .PEFM'JUUJOH model.fit(data, labels, epochs=10, batch_size=32)
  8. © 2018, Amazon Web Services, Inc. or Its Affiliates. All

    rights reserved. 9DFQUJPO৬ೣԋೞח
 ݽ؛ઁ੘৘द
  9. 9DFQUJPOJO,FSBT • ,FSBTীղ੢غযҍ߄۽ࢎਊоמ ੗زਵ۽ݽ؛׮਍۽٘ from tensorflow.python.keras.applications.xception import Xception from tensorflow.python.keras.preprocessing

    import image from tensorflow.python.keras.applications.resnet50 import preprocess_input, decode_predictions import numpy as np model = Xception(weights='imagenet') img_path = 'elephant.jpg' img = image.load_img(img_path, target_size=(224, 224)) x = image.img_to_array(img) x = np.expand_dims(x, axis=0) x = preprocess_input(x) preds = model.predict(x) print('Predicted:', decode_predictions(preds, top=3)[0])
  10. © 2018, Amazon Web Services, Inc. or Its Affiliates. All

    rights reserved. ஖అҗೖ੗ܳҳ࠙ೞחݽ؛੉੓׮ݶ
  11. 9DFQUJPO.PEFM5SBJOJOH৘द • ஖అࢎ૓੢җೖ੗ࢎ૓੢ ೟णਊ  • ஖అࢎ૓੢җೖ੗ࢎ૓੢ పझ౟ਊ  •

    9DFQUJPOݽ؛ӝ߈5SBOTGFS-FBSOJOH from tensorflow.python.keras.applications.xception import Xception model = Xception(include_top=False, weights='imagenet') 'VMM$POOFDUFE -BZFSઁѢ 5SBOTGFS-FBSOJOH
  12. ౟ۨ੉׬ೣࣻ def train(train_data_dir, validation_data_dir, model_path): base_model = Xception(weights='imagenet', include_top=False) x

    = base_model.output x = GlobalAveragePooling2D()(x) predictions = Dense(nb_classes, activation='softmax')(x) model = Model(base_model.input, predictions) transformation_ratio = .05 train_datagen = ImageDataGenerator(rescale=1. / 255, rotation_range=transformation_ratio, shear_range=transformation_ratio, zoom_range=transformation_ratio, cval=transformation_ratio, horizontal_flip=True, vertical_flip=True) पઁ೟णदఆݽ؛ ౟ۨ੉׬ࣇ੉޷૑ܳ߸ഋ೧ؘ੉ఠ۝ૐ಩ 9DFQUJPO߬੉झ
  13. ౟ۨ੉׬ೣࣻ def train(train_data_dir, validation_data_dir, model_path): ...(ࢤۚ)... train_generator = train_datagen.flow_from_directory(train_data_dir, batch_size=32,

    class_mode='categorical') validation_generator = validation_datagen.flow_from_directory(validation_data_dir, batch_size=32, class_mode='categorical') model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy']) top_weights_path = os.path.join(os.path.abspath(model_path), 'top_model_weights.h5') callbacks_list = [ ModelCheckpoint(top_weights_path, monitor='val_acc', verbose=1, save_best_only=True), EarlyStopping(monitor='val_acc', patience=5, verbose=0) ] model.fit_generator(train_generator, samples_per_epoch=train_generator.nb_sample, nb_epoch=nb_epoch / 5, validation_data=validation_generator, nb_val_samples=validation_generator.nb_sample, callbacks=callbacks_list) ૐ಩ؘ੉ఠࢤࢿӝ ݽ؛$PNQJMF ݽ؛5SBJOJOH
  14. ౟ۨ੉׬ਸ݃஖Ҋ • ݽ؛౟ۨ੉׬ਸ݃஖ݶKTPO౵ੌҗI౵ੌ۽&YQPSU
 ппݽ؛җݽ؛੄о઺஖  • पઁ*OGFSFODFܳ૓೯ೡٸח
 ਤীࢲ&YQPSUೠݽ؛җݽ؛о઺஖݅ਸ੍যࢎਊ final_weights_path =

    os.path.join(os.path.abspath(model_path), 'model_weights.h5') model.save_weights(final_weights_path) model_json = model.to_json() json_file = open(os.path.join(os.path.abspath(model_path), 'model.json'), 'w') json_file.write(model_json) ݽ؛੷੢ XFJHIU੷੢
  15. ݽ؛5FTU"DDVSBDZஏ੿ೞӝ def inference(trained_model_dir, test_data_dir, results_path): # load json and create

    model json_file = open(os.path.join(trained_model_dir, model_name), 'r') loaded_model_json = json_file.read() json_file.close() model = model_from_json(loaded_model_json) model.load_weights(os.path.join(trained_model_dir, model_weights)) # Read Data test_datagen = ImageDataGenerator(rescale=1. / 255) test_generator = test_datagen.flow_from_directory(test_data_dir, batch_size=batch_size, shuffle=False) # Calculate class posteriors probabilities y_probabilities = model.predict_generator(test_generator, val_samples=test_generator.nb_sample) # Calculate class labels y_classes = probas_to_classes(y_probabilities) filenames = [filename.split('/')[1] for filename in test_generator.filenames] ids = [filename.split('.')[0] for filename in filenames] ݽ؛੍ӝ о઺஖੍ӝ
  16. © 2018, Amazon Web Services, Inc. or Its Affiliates. All

    rights reserved. ٩۞׬ݽ؛ਸ
 -BNCEBীࢲج۰ࠁҊरযਃ
  17. © 2018, Amazon Web Services, Inc. or Its Affiliates. All

    rights reserved. Ӓր&$ীࢲجܻݶউغաਃ  ৵-BNCEBܳॳաਃ
  18. © 2018, Amazon Web Services, Inc. or Its Affiliates. All

    rights reserved. -BNCEBWT&$ ঱ઁ ޖ঺ਸॄঠೞա
  19. -BNCEBWT&$ • زद୊ܻܳрಞೞѱ"84ױীࢲ੸ਊоמ
 ௏٘ࣻ੿੉೙ਃೞ૑ঋ਺  • ۈ׮ೣࣻח੹୓ೣࣻী؀೧3FHJPO߹
 زदীѐೣ߽ࣻ۳प೯оמ • "844VQQPSU$FOUFSܳా೧زद୊ܻ۝ૐооמ

    • $16৬(16ীٮۄݽ؛੄ࣘبର੉о௼૑ঋ਷҃਋
 *NBHF/FU*OGFSFODF.PEFM 0 
 /FVSBM4UZMF5SBOTGFS 9 
 
 (16۽بוؘܽ दрت-BNCEBח9
  20. © 2018, Amazon Web Services, Inc. or Its Affiliates. All

    rights reserved. 9DFQUJPOݽ؛*OGFSFODFೣࣻ  -BNCEBೣࣻ۽ٜ݅ӝ
  21. # event ё୓ { 'Records': [ { 'eventVersion': '2.0', 'eventSource':

    'aws:s3', 'awsRegion': 'ap-northeast-2', # ߡఉ ܻ੹ 'eventTime': '2017-12-13T03:28:13.528Z', # স۽٘ ৮ܐ दп 'eventName': 'ObjectCreated:Put', 'userIdentity': {'principalId': 'AFK2RA1O3ML1F'}, 'requestParameters': {'sourceIPAddress': '123.24.137.5'}, 'responseElements': { 'x-amz-request-id': '1214K424C14C384D', 'x-amz-id-2': 'BOTBfAoB/gKBbn412ITN4t2psTW499iMRKZDK/CQTsjrkeSSzSdsDUMGabcdnvHeYNtbTDHoHKs=' }, 's3': { 's3SchemaVersion': '1.0', 'configurationId': 'b249eeda-3d48-4319-a7e2-853f964c1a25', 'bucket': { 'name': 'aws-summit-kr-2018', # ߡఉ ੉ܴ 'ownerIdentity': { 'principalId': 'AFK2RA1O3ML1F' }, 'arn': 'arn:aws:s3:::aws-summit-kr-2018' }, 'object': { 'key': 'img/test_img.png', # ߡఉ ղ ౵ੌ੄ ҃۽ 'size': 11733, # ౵ੌ ௼ӝ 'eTag': 'f2d12d123aebda1cc1fk17479207e838', 'sequencer': '125B119E4D7B2A0A48' } } } ] } #VDLFU /BNF 'JMF1BUI JO#VDLFU
  22. 4ীࢲ౵ੌ׮਍۽٘স۽٘ • #PUPਸ੉ਊ೧౵ੌ׮਍۽٘߂স۽٘ ACCESS_KEY = os.environ.get('ACCESS_KEY') SECRET_KEY = os.environ.get('SECRET_KEY') def

    downloadFromS3(strBucket, s3_path, local_path): s3_client = boto3.client('s3', aws_access_key_id=ACCESS_KEY, aws_secret_access_key=SECRET_KEY) s3_client.download_file(strBucket, s3_path, local_path) def uploadToS3(bucket, s3_path, local_path): s3_client = boto3.client('s3', aws_access_key_id=ACCESS_KEY, aws_secret_access_key=SECRET_KEY) s3_client.upload_file(local_path, bucket, s3_path) -BNCEBী޷ܻജ҃߸ࣻ۽"84ఃद௼݁١۾
  23. • খࢲ݅ٚIBOEMFSೣࣻղ੉޷૑׮਍۽٘୶о • AUNQAಫ؊ղ੷੢ .ઁೠ 
 ഑਷ৡݫݽܻ'JMF0CKFDU۽ҳࢿ def handler(event, context):

    bucket_name = event['Records'][0]['s3']['bucket']['name'] file_path = event['Records'][0]['s3']['object']['key'] file_name = file_path.split('/')[-1] downloadFromS3(bucket_name, file_path, '/tmp/'+file_name) 4ীࢲ౵ੌ׮਍۽٘স۽٘ -BNCEBীࢲ'JMFXSJUFח
 AUNQAղীࢲ݅оמ ׮ܲҔীॳݶ*0&SSPSߊࢤ
  24. 9DFQUJPOݽ؛׮਍۽٘ • 4ߡఉীࢲݽ؛XFJHIU౵ੌ I ׮਍۽٘ def handler(event, context): bucket_name =

    event['Records'][0]['s3']['bucket']['name'] file_path = event['Records'][0]['s3']['object']['key'] file_name = file_path.split('/')[-1] downloadFromS3(bucket_name, file_path, '/tmp/'+file_name) downloadFromS3( 'aws-summit-kr-2018', 'xception_weights_tf_dim_ordering_tf_kernels.h5', '/tmp/.keras/xception_weights_tf_dim_ordering_tf_kernels.h5' ) ,FSBTח_LFSBT৬UNQLFSBTفҔীࢲ౵ੌਸ଺ইࢎਊೞ૑݅  -BNCEBীࢲח_LFSBTীॳӝӂೠ੉হਵ޲۽UNQLFSBTܳࢎਊ
  25. 9DFQUJPO1SFEJDUೣࣻ • AQSFEJDUAۄח੉ܴਵ۽9DFQUJPOݽ؛ࢎਊೞӝ from tensorflow.python.keras.applications.xception import Xception from tensorflow.python.keras.preprocessing import

    image from tensorflow.python.keras.applications.resnet50 import preprocess_input, decode_predictions import numpy as np def predict(img_path): model = Xception(weights='imagenet') img = image.load_img(img_path, target_size=(224, 224)) x = image.img_to_array(img) x = np.expand_dims(x, axis=0) x = preprocess_input(x) preds = model.predict(x) return decode_predictions(preds, top=3)[0] ੉޷૑҃۽ܳੋ੗۽߉ই1SFEJDUܳ݃஘Ѿҗчਸ߈ജ ױࣽೠѾҗч3FUVSO؀न %ZOBNP%#١ী੸੤੉޷૑࠙ܨറಫ؊߹4স۽٘
  26. ׮दೠߣ-BNCEBݫੋೣࣻܳࣻ੿ೞݶ • 4&WFOU%PXOMPBE*NBHF.PEFM1SFEJDU def handler(event, context): bucket_name = event['Records'][0]['s3']['bucket']['name'] file_path

    = event['Records'][0]['s3']['object']['key'] file_name = file_path.split('/')[-1] downloadFromS3(bucket_name, file_path, '/tmp/'+file_name) downloadFromS3( 'aws-summit-kr-2018', 'xception_weights_tf_dim_ordering_tf_kernels.h5', '/tmp/.keras/xception_weights_tf_dim_ordering_tf_kernels.h5' ) result = predict('/tmp/'+file_name) return result
  27. © 2018, Amazon Web Services, Inc. or Its Affiliates. All

    rights reserved. -BNCEBস۽٘ਊ QBDL[JQٜ݅ӝ ೙ਃೠ1ZUIPO%FQFOEFODZٜ੉ ݽفݽৈ੓ח঑୷౵ੌٜ݅ӝ
  28. © 2018, Amazon Web Services, Inc. or Its Affiliates. All

    rights reserved. ੄ઓಁః૑о݆ইਃ
  29. © 2018, Amazon Web Services, Inc. or Its Affiliates. All

    rights reserved. $঱য҅ৌ੄ઓۄ੉࠳۞ܻ 1BOEBT/VNQZIQZ
  30. © 2018, Amazon Web Services, Inc. or Its Affiliates. All

    rights reserved. 04߹۽׮ܲXIM౵ੌ
  31. © 2018, Amazon Web Services, Inc. or Its Affiliates. All

    rights reserved. "84-BNCEB04 "NB[PO-JOVY
  32. -BNCEBPO"NB[PO-JOVY • ".*BN[OBNJIWNY@HQ • -JOVYழօBN[OY@ • CJU߄੉ցܻ݅૑ਗ • 1ZUIPO਷ࢎਊоמ
 #PUP

     CPUPDPSF  ࢸ஖غয੓਺ • $FOU04୊ۢ:VNࢎਊоמ ୭न੿ࠁ
 IUUQTEPDTBXTBNB[PODPNMBNCEBMBUFTUEHDVSSFOUTVQQPSUFEWFSTJPOTIUNM
  33. © 2018, Amazon Web Services, Inc. or Its Affiliates. All

    rights reserved. بழо೙ਃೞ׮
  34. © 2018, Amazon Web Services, Inc. or Its Affiliates. All

    rights reserved.  "NB[PO-JOVY
  35. %PDLFSীZVNਵ۽೙ਃಁః૑ࢸ஖ dev_install() { yum -y update yum -y upgrade yum

    install -y \ wget \ gcc \ gcc-c++ \ cmake \ python36-devel \ python36-virtualenv \ python36-pip \ findutils \ zlib-devel \ zip \ unzip \ blas-devel lapack-devel atlas-devel } ӝࠄಁః૑୭नসؘ੉౟ 1ZUIPOࢸ஖ ӝఋಁః૑ࢸ஖
  36. %PDLFSղWJSUVBMFOW۽೙ਃಁః૑݅ࢸ஖ mkvirtualenv() { cd /home/ rm -rf env python3 -m

    virtualenv env --python=python3 source env/bin/activate } pip_install() { source /home/env/bin/activate pip install -U pip wheel pip install --use-wheel tensorflow==1.7.0 --no-deps pip install protobuf html5lib bleach --no-deps pip install --use-wheel pillow==4.0.0 pip install h5py } FOWо࢚ജ҃ࢤࢿ ೙ਃ੄ઓࢿಁః૑ࢸ஖ OPEFQT੄ઓಁః૑ࢸ஖9 ೧׼ಁః૑݅ࢸ஖
  37. EFQFOEFODZ JOEFYQZQBDL[JQ gather_pack() { cd /home/ && rm -rf pack

    && mkdir pack && cd pack cp -R /home/env/lib/python3.6/site-packages/* . cp -R /home/env/lib64/python3.6/site-packages/* . cp /outputs/index.py /home/pack/index.py find . -type d -name "test" -exec rm -rf {} + find -name "*.so" | xargs strip find -name "*.so.*" | xargs strip rm -r pip && rm -r pip-* && rm -r wheel && rm -r wheel-* find . | grep -E "(__pycache__|\.pyc$)" | xargs rm -rf echo "stripped size $(du -sh /home/pack | cut -f1)" zip -FS -r1 /outputs/pack.zip * > /dev/null echo "compressed size $(du -sh /outputs/pack.zip | cut -f1)" } ঑୷ೡಫ؊ࢤࢿ ౵੉ॆۄ੉࠳۞ܻࠂࢎ पઁز੘ೞח JOEFYQZ 4IBSFE0CKFDU TP TUSJQ QZDBDIFઁѢ QBDL[JQਵ۽঑୷
  38. 46QMPBE-BNCEB'VODUJPOVQEBUFX#PUP import boto3 def upload_to_s3(bucket, s3_path, local_path): client = boto3.client('s3',

    aws_access_key_id=ACCESS_KEY,aws_secret_access_key=SECRET_KEY) client.upload_file(local_path, bucket, s3_path) def update_lambda(function_name, bucket, s3_path): client = boto3.client('lambda', aws_access_key_id=ACCESS_KEY,aws_secret_access_key=SECRET_KEY) client.update_function_code( FunctionName=function_name, S3Bucket=bucket, S3Key=s3_path, ) uploadToS3('ߡఉݺ', 'ߡఉղ҃۽/pack.zip', './pack.zip') update_lambda('ۈ׮ೣࣻ੉ܴ', 'ߡఉݺ', 'ߡఉղ҃۽/pack.zip') QBDL[JQস۽٘ MBNCEBೣࣻসؘ੉౟
  39. © 2018, Amazon Web Services, Inc. or Its Affiliates. All

    rights reserved. %FQFOEFODZܳݽف೤ଢ଼؊פ
 .#оֈযߡ۷যਃ
  40. %FQFOEFODZଂѐӝ def download_s3_object(strBucket, s3_path): import io file_obj = io.BytesIO() s3_client

    = boto3.client( 's3', aws_access_key_id=ACCESS_KEY, aws_secret_access_key=SECRET_KEY ) s3_client.download_fileobj(strBucket, s3_path, file_obj) return file_obj pack2 = download_s3_object('bucket-name', 'pack2.zip') import zipfile zip_ref = zipfile.ZipFile(pack2) zip_ref.extractall('/tmp') zip_ref.close() import sys sys.path.append("/tmp") ౵ੌਸ׮਍߉૑݅ ৡݫݽܻ੄#ZUFT*0ী੷੢ UNQಫ؊ী %FQFOEFODZ঑୷೧ઁ пઙۄ੉࠳۞ܻJNQPSU੹ JNQPSU1"5)୶о
  41. • ٩۞׬ݽ؛җ%FQFOEFODZ౵ੌ਷য٣ࢲоઉয়ա 4 • 4ীࢲоઉয়ݶ੹࣠࠺ਊ਷হաਃ 
 э਷3FHJPOղ੹࣠਷੹࣠࠺ਊ੉ߊࢤೞ૑ঋणפ׮ • 4ীࢲоઉয়חؘदр੉ٜ૑ঋաਃ 


    -BNCEBदр੉ઑӘ؊য়ېѦܻ૑݅ 
 ௾ਊ۝੄ݽ؛౵ੌਸࢎਊೡࣻ੓׮ח੼੉੓णפ׮ • ਊ۝ઁೠ਷হաਃ 
 पઁ౵ੌ۽ॵ׮ݶ୭؀.# ৡݫݽܻ۽୭؀(੉ղ पઁ۽ࢎਊೞӝীޙઁ੓૑ঋաਃ
  42. © 2018, Amazon Web Services, Inc. or Its Affiliates. All

    rights reserved. -BNCEB 4 "1*(BUFXBZ
 8FC4FSWJDF
  43. © 2018, Amazon Web Services, Inc. or Its Affiliates. All

    rights reserved. -BNCEB5SJHHFSJOH &WFOU
  44. © 2018, Amazon Web Services, Inc. or Its Affiliates. All

    rights reserved. ঒ Ӓؘ۠4ী ੉޷૑ܳযڌѱৢܻ૑
  45. © 2018, Amazon Web Services, Inc. or Its Affiliates. All

    rights reserved. 41045 "1*(BUFXBZ पઁ স۽٘ স۽٘੹ ӂೠࠗৈ
  46. © 2018, Amazon Web Services, Inc. or Its Affiliates. All

    rights reserved. ױࣽೞѱ-BNCEBജ҃ীࢲ प೯݅೧ࠁҊर׮ݶ
  47. © 2018, Amazon Web Services, Inc. or Its Affiliates. All

    rights reserved. docker run --rm -v "$PWD":/var/task lambci/lambda:python3.6 my_module.my_handler ױࣽ൤ೣࣻܳप೯ೞ۰ݶ &WFOUੋ੗ܳ੹׳ೞ۰ݶ docker run --rm -v "$PWD":/var/task lambci/lambda:python3.6 my_module.my_handler '{"some": "event"}'
  48. © 2018, Amazon Web Services, Inc. or Its Affiliates. All

    rights reserved. ѱ੐ഥࢎীࢲ٩۞׬
  49. © 2018, Amazon Web Services, Inc. or Its Affiliates. All

    rights reserved. ࢎۈ੄׀ਸ؀न೧ࠁ੗
  50. © 2018, Amazon Web Services, Inc. or Its Affiliates. All

    rights reserved. ࢎۈ੉ࠁ૑ޅೠ1BUUFSOਸ଺੗
  51. © 2018, Amazon Web Services, Inc. or Its Affiliates. All

    rights reserved. $੉਄੉ইצ઴ইפ
  52. © 2018, Amazon Web Services, Inc. or Its Affiliates. All

    rights reserved. ױࣽೠܙ߬੉झࠗఠ ࠂ੟ೠ٩۞׬ө૑
  53. © 2018, Amazon Web Services, Inc. or Its Affiliates. All

    rights reserved. ਋ܻэ੉ੌ೧ਃ 8FSF)JSJOH
  54. 2O"