Ê×Ò³
ѧϰ
»î¶¯
רÇø
¹¤¾ß
TVP
·¢²¼
¾«Ñ¡ÄÚÈÝ/¼¼ÊõÉçȺ/ÓŻݲúÆ·,¾¡ÔÚС³ÌÐò
Á¢¼´Ç°Íù

Netflix¿ªÔ´ÄÚ²¿Python¹¤¾ßMetaflow£¬²¿ÊðЧÂÊÌáÉý17±¶

Netflix¾­³£ÒÔ¿ª·ÅÔ´´úÂëµÄÐÎʽÏò¹«ÖÚ·¢²¼ÆäÄÚ²¿¹¤¾ß¡£½üÈÕ£¬NetflixµÄÊý¾Ý¿ÆѧÍŶÓÒѽ«ÆäMetaflow Python¿â¿ªÔ´£¬¸Ã¿âÊÇ¡°ÒÔÈËΪÖÐÐÄ¡±µÄ»úÆ÷ѧϰ»ù´¡¼Ü¹¹µÄ¹Ø¼ü²¿·Ö£¬ÓÃÓÚ¹¹½¨ºÍ²¿ÊðÊý¾Ý¿Æѧ¹¤×÷Á÷¡£

MetaflowÓëApache AirflowºÍLuigiµÈÏîÄ¿ÓÐÐí¶àÏàËƵĸÅÄ¿ª·¢ÈËÔ±¿ÉÄܶԴËÓÐÒ»¶¨µÄ¾­Ñé¡£ÀýÈ磬ʹÓÃMetaflow£¬Äã¿ÉÒÔ´´½¨ºÍÖ´ÐÐDAGs(ÓÐÏòÎÞ»·Í¼)¡£

´úÂë½á¹¹

ÆäÖÐÒ»¸ö¹Ø¼üÌØÐÔÊÇMetaflow×Ô¶¯½«ËùÓдúÂë¡¢Êý¾ÝºÍÒÀÀµÏî¿ìÕÕµ½Amazon S3µÄÄÜÁ¦¡£ÓÐÁËÕâ¸ö¹¦ÄÜÌØÐÔ£¬Ê¹MetaflowÄܹ»ÊµÏÖ×Ô¶¯°æ±¾¿ØÖƺÍʵÑé¸ú×Ù£¬Òò´Ë¿ª·¢ÈËÔ±¿ÉÒÔ°²È«µØ¼ì²éºÍ»Ö¸´MetaflowµÄÖ´ÐÐÇé¿ö¡£

ÍⲿPython¿â

PythonÒѾ­³ÉΪÊý¾Ý¿ÆѧµÄ±ê×¼ÓïÑÔÖ®Ò»£¬Òò´ËËüÓÐÐí¶à³£ÓõĿ⡣MetaflowÔÊÐíµ¼ÈëÍⲿ¿â£¬²¢Ö§³ÖËùÓй«¹²»úÆ÷ѧϰ¿ò¼Ü¡£

Òò´Ë£¬Äã¿ÉÒÔʹÓÃËùÓÐÄãÊìϤµÄÊý¾Ý¿Æѧ¿â£¬ÀýÈçPyTorchºÍTensorFlow¡£Ö»ÐèʹÓùßÓõÄPython´úÂë±àдģÐÍ¡£

´ËÍ⣬NetflixÔÚ2Ô·ݵÄÒ»³¡Ñݽ²ÖÐ͸¶£¬MetaflowÒѾ­½«Netflix»úÆ÷ѧϰÏîÄ¿µÄ²¿Êðʱ¼äÖÐλÊý´ÓËĸöÔÂËõ¶Ìµ½Á˽ö½ö7Ì죬²¿ÊðЧÂÊ´ó´óÌá¸ß

AWS¼¯³É

¶àÄêÀ´£¬NetflixÒ»Ö±ÊÇʹÓÃAWSµÄ×î´óÆóÒµÖ®Ò»£¬Òò´ËMetaflowÓëÖÚ¶àAWS·þÎñ¼¯³ÉÔÚÒ»Æð¾Í²»×ãΪÆæÁË£¬´ÓÎĵµÖпÉÒÔ¿´µ½£¬ÕâЩÊǵ±Ç°µÄ¼¯³É£¨½«À´»¹»áÓÐÁ½¸ö¼¯³É£©£º

½èÖúÄÚÖõÄÇ¿´óS3¿Í»§¶Ë£¬Metaflow¿ÉÒÔ¼ÓÔظߴï10GbpsµÄÊý¾Ý¡£AWSÉÏMetaflow µÄÎĵµµØÖ·£ºhttps://docs.metaflow.org/metaflow-on-aws/metaflow-on-aws

Metaflow»¹Óë»ùÓÚAWSÈÝÆ÷µÄ¼ÆËãƽ̨Batch¼¯³É¡£ Netflix ÈÏΪ£¬AWSÉϵÄMetaflow¿ÉÒÔÈÿª·¢ÈËÔ±Ìá¸ßÔڱʼDZ¾µçÄÔÉϽøÐпª·¢µÄËٶȣ¬Í¬Ê±¿ÉÒÔÔÚÔÆÖÐʹÓøüÉîµÄ¼ÆËã×ÊÔ´¡£ ?MetaflowʹµÃ²»±ØÔÚÿÖÖ״̬϶¼¶Ô´úÂë»ò¿â½øÐиü¸Ä£¬´Ó¶øʹÔÚ±¾µØºÍÔ¶³ÌÖ´ÐÐģʽ֮¼äÇáËɵØÀ´»ØÒƶ¯£¬Õâ·´¹ýÀ´ÓÖʹ¹ÊÕÏÅųý¸ü¼ÓÈÝÒס£

MetaflowµÄÔ´´úÂë¿ÉÔÚGitHubÉÏ»ñµÃ£¬Ä¿Ç°ÒѾ­ÔÚGithubÉϱêÐÇ1.4K£¬76¸öFork£¨GithubµØÖ·£ºhttps://github.com/Netflix/metaflow£©

  • ·¢±íÓÚ:
  • Ô­ÎÄÁ´½Ó£ºhttp://news.51cto.com/art/201912/607420.htm
  • ÈçÓÐÇÖȨ£¬ÇëÁªÏµ cloudcommunity@tencent.com ɾ³ý¡£

Ïà¹Ø¿ìѶ

ɨÂë

Ìí¼ÓÕ¾³¤ ½ø½»Á÷Ⱥ

ÁìȡרÊô 10ÔªÎÞÃż÷ȯ

˽Ïí×îР¼¼Êõ¸É»õ

ɨÂë¼ÓÈ뿪·¢ÕßÉçȺ
Áìȯ
http://www.vxiaotou.com