Skip to main content
 首页 » 编程设计

python之如何从 Python 连接到 H2O Deep Water Docker Image

2025年02月15日26arxive

抱歉,又是一个真正的初学者问题:

我按照深水手册 (http://docs.h2o.ai/h2o/latest-stable/h2o-docs/booklets/DeepWaterBooklet.pdf) 中的说明下载了 H2O 深水 Docker 容器:

docker run -it --rm -p 54321:54321 -p 8080:8080 -v $PWD:/host opsh2oai/h2o-deepwater-cpu 
 
java -jar /opt/h2o.jar 

我成功启动了 H2O,并且可以在我的本地浏览器中通过 localhost:54321 使用 Flow UI 访问它。

我想做上面小册子中的 MNIST Python 示例。我将 Anaconda 和 Spyder IDE 与 Python 2.7 结合使用。

我想不是使用 init(),而是假设我必须使用 connect() 方法。无论哪种方式都不起作用,我得到超时、连接错误或 HTTP 异常,具体取决于我尝试的内容...我尝试了几个 IP(包括本地主机),例如:

from h2o.estimators.deepwater import H2ODeepWaterEstimator 
# Start or connect to H2O 
#h2o.init() 
h2o.connect(ip="127.0.0.1", port=54321) 

我的启动日志显示如下。

root@d54b0501b7c1:/# java -jar /opt/h2o.jar 
08-11 08:46:32.904 172.17.0.2:54321      9      main      INFO: Found XGBoost backend with library: xgboost4j 
08-11 08:46:32.926 172.17.0.2:54321      9      main      INFO: ----- H2O started  ----- 
08-11 08:46:32.928 172.17.0.2:54321      9      main      INFO: Build git branch: (HEAD detached at 0c917c766) 
08-11 08:46:32.928 172.17.0.2:54321      9      main      INFO: Build git hash: 0c917c766f28860a19ff3f110e1f405eb9f41623 
08-11 08:46:32.928 172.17.0.2:54321      9      main      INFO: Build git describe: jenkins-master-3971 
08-11 08:46:32.928 172.17.0.2:54321      9      main      INFO: Build project version: 3.13.0.356 (latest version: 3.14.0.1) 
08-11 08:46:32.928 172.17.0.2:54321      9      main      INFO: Build age: 12 days 
08-11 08:46:32.929 172.17.0.2:54321      9      main      INFO: Built by: 'jenkins' 
08-11 08:46:32.929 172.17.0.2:54321      9      main      INFO: Built on: '2017-07-29 13:31:58' 
08-11 08:46:32.929 172.17.0.2:54321      9      main      INFO: Watchdog Build git branch: (unknown) 
08-11 08:46:32.929 172.17.0.2:54321      9      main      INFO: Watchdog Build git hash: (unknown) 
08-11 08:46:32.929 172.17.0.2:54321      9      main      INFO: Watchdog Build git describe: (unknown) 
08-11 08:46:32.929 172.17.0.2:54321      9      main      INFO: Watchdog Build project version: (unknown) 
08-11 08:46:32.930 172.17.0.2:54321      9      main      INFO: Watchdog Built by: (unknown) 
08-11 08:46:32.930 172.17.0.2:54321      9      main      INFO: Watchdog Built on: (unknown) 
08-11 08:46:32.930 172.17.0.2:54321      9      main      INFO: XGBoost Build git branch: (unknown) 
08-11 08:46:32.930 172.17.0.2:54321      9      main      INFO: XGBoost Build git hash: (unknown) 
08-11 08:46:32.930 172.17.0.2:54321      9      main      INFO: XGBoost Build git describe: (unknown) 
08-11 08:46:32.930 172.17.0.2:54321      9      main      INFO: XGBoost Build project version: (unknown) 
08-11 08:46:32.931 172.17.0.2:54321      9      main      INFO: XGBoost Built by: (unknown) 
08-11 08:46:32.931 172.17.0.2:54321      9      main      INFO: XGBoost Built on: (unknown) 
08-11 08:46:32.931 172.17.0.2:54321      9      main      INFO: Processed H2O arguments: [] 
08-11 08:46:32.931 172.17.0.2:54321      9      main      INFO: Java availableProcessors: 2 
08-11 08:46:32.931 172.17.0.2:54321      9      main      INFO: Java heap totalMemory: 31.0 MB 
08-11 08:46:32.932 172.17.0.2:54321      9      main      INFO: Java heap maxMemory: 444.5 MB 
08-11 08:46:32.932 172.17.0.2:54321      9      main      INFO: Java version: Java 1.8.0_131 (from Oracle Corporation) 
08-11 08:46:32.932 172.17.0.2:54321      9      main      INFO: JVM launch parameters: [] 
08-11 08:46:32.932 172.17.0.2:54321      9      main      INFO: OS version: Linux 4.9.36-moby (amd64) 
08-11 08:46:32.932 172.17.0.2:54321      9      main      INFO: Machine physical memory: 1.95 GB 
08-11 08:46:32.932 172.17.0.2:54321      9      main      INFO: X-h2o-cluster-id: 1502441188713 
08-11 08:46:32.933 172.17.0.2:54321      9      main      INFO: User name: 'root' 
08-11 08:46:32.933 172.17.0.2:54321      9      main      INFO: IPv6 stack selected: false 
08-11 08:46:32.933 172.17.0.2:54321      9      main      INFO: Possible IP Address: eth0 (eth0), 172.17.0.2 
08-11 08:46:32.933 172.17.0.2:54321      9      main      INFO: Possible IP Address: lo (lo), 127.0.0.1 
08-11 08:46:32.933 172.17.0.2:54321      9      main      INFO: H2O node running in unencrypted mode. 
08-11 08:46:32.943 172.17.0.2:54321      9      main      INFO: Internal communication uses port: 54322 
08-11 08:46:32.943 172.17.0.2:54321      9      main      INFO: Listening for HTTP and REST traffic on http://172.17.0.2:54321/ 
08-11 08:46:32.945 172.17.0.2:54321      9      main      INFO: H2O cloud name: 'root' on /172.17.0.2:54321, discovery address /225.53.128.226:57653 
08-11 08:46:32.946 172.17.0.2:54321      9      main      INFO: If you have trouble connecting, try SSH tunneling from your local machine (e.g., via port 55555): 
08-11 08:46:32.946 172.17.0.2:54321      9      main      INFO:   1. Open a terminal and run 'ssh -L 55555:localhost:54321 root@172.17.0.2' 
08-11 08:46:32.946 172.17.0.2:54321      9      main      INFO:   2. Point your browser to http://localhost:55555 
08-11 08:46:32.955 172.17.0.2:54321      9      main      INFO: Log dir: '/tmp/h2o-root/h2ologs' 
08-11 08:46:32.957 172.17.0.2:54321      9      main      INFO: Cur dir: '/' 
08-11 08:46:33.072 172.17.0.2:54321      9      main      INFO: HDFS subsystem successfully initialized 
08-11 08:46:33.085 172.17.0.2:54321      9      main      INFO: S3 subsystem successfully initialized 
08-11 08:46:33.088 172.17.0.2:54321      9      main      INFO: Flow dir: '/root/h2oflows' 
08-11 08:46:33.142 172.17.0.2:54321      9      main      INFO: Cloud of size 1 formed [/172.17.0.2:54321] 
08-11 08:46:33.166 172.17.0.2:54321      9      main      INFO: Registered parsers: [GUESS, ARFF, XLS, SVMLight, AVRO, PARQUET, CSV] 
08-11 08:46:33.167 172.17.0.2:54321      9      main      INFO: Watchdog extension initialized 
08-11 08:46:33.168 172.17.0.2:54321      9      main      INFO: XGBoost extension initialized 
08-11 08:46:33.171 172.17.0.2:54321      9      main      INFO: Registered 2 core extensions in: 341ms 
08-11 08:46:33.172 172.17.0.2:54321      9      main      INFO: Registered H2O core extensions: [Watchdog, XGBoost] 
08-11 08:46:33.184 172.17.0.2:54321      9      main      INFO: Found XGBoost backend with library: xgboost4j 
08-11 08:46:33.702 172.17.0.2:54321      9      main      INFO: Registered: 160 REST APIs in: 529ms 
08-11 08:46:33.711 172.17.0.2:54321      9      main      INFO: Registered REST API extensions: [XGBoost, Algos, AutoML, Core V3, Core V4] 
08-11 08:46:34.073 172.17.0.2:54321      9      main      INFO: Registered: 230 schemas in 354ms 
08-11 08:46:34.073 172.17.0.2:54321      9      main      INFO: H2O started in 5343ms 
08-11 08:46:34.073 172.17.0.2:54321      9      main      INFO:  
08-11 08:46:34.075 172.17.0.2:54321      9      main      INFO: Open H2O Flow in your web browser: http://172.17.0.2:54321 

在我的 Mac/Docker 容器中连接到这个 H2O Deep Water Docker 容器的正确 Python 代码或缺少的配置是什么?

似乎是安全/Docker 配置问题???

h2o.connect(ip="127.0.0.1", port=54321) 
Connecting to H2O server at http://127.0.0.1:54321...Traceback (most recent call last): 
 
  File "<ipython-input-12-e78edc3b0d61>", line 1, in <module> 
    h2o.connect(ip="127.0.0.1", port=54321) 
 
  File "/Users/kai.waehner/anaconda/lib/python2.7/site-packages/h2o/h2o.py", line 74, in connect 
    cluster_id=cluster_id, cookies=cookies, verbose=verbose) 
 
  File "/Users/kai.waehner/anaconda/lib/python2.7/site-packages/h2o/backend/connection.py", line 175, in open 
    conn._cluster = conn._test_connection(retries, messages=_msgs) 
 
  File "/Users/kai.waehner/anaconda/lib/python2.7/site-packages/h2o/backend/connection.py", line 414, in _test_connection 
    cld = self.request("GET /3/Cloud") 
 
  File "/Users/kai.waehner/anaconda/lib/python2.7/site-packages/h2o/backend/connection.py", line 259, in request 
    return self._process_response(resp, save_to) 
 
  File "/Users/kai.waehner/anaconda/lib/python2.7/site-packages/h2o/backend/connection.py", line 574, in _process_response 
    data = response.json(object_pairs_hook=H2OResponse) 
 
  File "/Users/kai.waehner/anaconda/lib/python2.7/site-packages/requests/models.py", line 877, in json 
    self.content.decode(encoding), **kwargs 
 
  File "/Users/kai.waehner/anaconda/lib/python2.7/json/__init__.py", line 352, in loads 
    return cls(encoding=encoding, **kw).decode(s) 
 
  File "/Users/kai.waehner/anaconda/lib/python2.7/json/decoder.py", line 364, in decode 
    obj, end = self.raw_decode(s, idx=_w(s, 0).end()) 
 
  File "/Users/kai.waehner/anaconda/lib/python2.7/json/decoder.py", line 380, in raw_decode 
    obj, end = self.scan_once(s, idx) 
 
  File "/Users/kai.waehner/anaconda/lib/python2.7/site-packages/h2o/backend/connection.py", line 691, in __new__ 
    if schema == "CloudV3": return H2OCluster.from_kvs(keyvals) 
 
  File "/Users/kai.waehner/anaconda/lib/python2.7/site-packages/h2o/backend/cluster.py", line 47, in from_kvs 
    raise AttributeError("Attribute %s cannot be set on H2OCluster (= %r)" % (k, v)) 
 
AttributeError: Attribute internal_security_enabled cannot be set on H2OCluster (= False) 

感谢您的帮助。

请您参考如下方法:

当您在 docker 镜像中启动 h2o 并使用 -p 54321:54321 时,h2o 将在运行 docker 的机器上的 localhost:54321 运行。然后您可以使用 h2o.init() 以正常方式连接。当 init 启动时,它会寻找正在运行的 h2o(您随后拥有)并连接到它。