我成功地遵循了DAVID TAIEB的这个伟大的tutorial。问题是图形需要这么长的时间来更改.It大约需要几分钟,而不是几秒钟,因为它应该是!!
可能的问题是什么?
当我在Scala Notebook中运行此单元时:
demo.startTwitterStreaming(sc)输出如下所示:
Registering JaasConfiguration: /tmp/1sFcnUVoabfuBAyF/jaas.conf
Setting hadoop configuration for swift container
https://kafka-rest-prod01.messagehub.services.us-south.bluemix.net:443/admin/topics
https://kafka-rest-prod01.messagehub.services.us-south.bluemix.net:443/admin/topics
default location of ssl Trust store is: /usr/local/src/spark160master/ibm-java-x86_64-80/jre/lib/security/cacerts
Twitter stream started
Tweets are collected real-time and analyzed
To stop the streaming and start interacting with the data use: StreamingTwitter.stopTwitterStreaming
Receiver Started: KafkaReceiver-0
Batch started with 0 records
-------------------------------------------
Time: 1460572070000 ms
-------------------------------------------
Batch completed with 0 records
Batch started with 0 records
-------------------------------------------
Time: 1460572075000 ms
-------------------------------------------
Batch completed with 0 records
Batch started with 487 records
-------------------------------------------
Time: 1460572080000 ms
-------------------------------------------
(#TDS,(1,List(agreeableness_big5, analytical, anger, confident, conscientiousness_big5, disgust, extraversion_big5, fear, joy, neuroticism_big5, openness_big5, sadness, tentative),List(30.5, 0.0, 16.5, 0.0, 19.5, 3.0, 38.5, 20.0, 5.0, 32.0, 28.0, 9.0, 0.0)))
(#DolceAmoreILoveYou,(1,List(agreeableness_big5, analytical, anger, confident, conscientiousness_big5, disgust, extraversion_big5, fear, joy, neuroticism_big5, openness_big5, sadness, tentative),List(5.0, 0.0, 6.0, 0.0, 17.0, 1.0, 17.5, 12.5, 24.5, 34.0, 44.0, 2.0, 0.0)))
(#nasty,(1,List(agreeableness_big5, analytical, anger, confident, conscientiousness_big5, disgust, extraversion_big5, fear, joy, neuroticism_big5, openness_big5, sadness, tentative),List(0.5, 0.0, 16.0, 0.0, 0.5, 34.5, 49.0, 1.0, 0.0, 46.0, 16.0, 2.5, 0.0)))
(#Job:,(1,List(agreeableness_big5, analytical, anger, confident, conscientiousness_big5, disgust, extraversion_big5, fear, joy, neuroticism_big5, openness_big5, sadness, tentative),List(8.0, 0.0, 2.5, 0.0, 28.0, 1.5, 4.5, 2.5, 45.0, 19.5, 37.0, 3.5, 0.0)))
(#bigbrand,(1,List(agreeableness_big5, analytical, anger, confident, conscientiousness_big5, disgust, extraversion_big5, fear, joy, neuroticism_big5, openness_big5, sadness, tentative),List(5.5, 0.0, 6.0, 0.0, 49.0, 4.0, 2.0, 15.0, 12.5, 3.5, 48.5, 5.5, 0.0)))
(#SwitchUp,(1,List(agreeableness_big5, analytical, anger, confident, conscientiousness_big5, disgust, extraversion_big5, fear, joy, neuroticism_big5, openness_big5, sadness, tentative),List(47.5, 0.0, 3.5, 0.0, 34.5, 1.0, 47.0, 29.0, 11.5, 22.5, 1.0, 3.5, 0.0)))
(#Hiring,(1,List(agreeableness_big5, analytical, anger, confident, conscientiousness_big5, disgust, extraversion_big5, fear, joy, neuroticism_big5, openness_big5, sadness, tentative),List(8.0, 0.0, 2.5, 0.0, 28.0, 1.5, 4.5, 2.5, 45.0, 19.5, 37.0, 3.5, 0.0)))
(#entrepreneurship,(1,List(agreeableness_big5, analytical, anger, confident, conscientiousness_big5, disgust, extraversion_big5, fear, joy, neuroticism_big5, openness_big5, sadness, tentative),List(20.5, 25.5, 5.0, 0.0, 28.0, 9.0, 12.0, 3.0, 28.0, 4.0, 12.5, 2.5, 0.0)))
(#startup,(1,List(agreeableness_big5, analytical, anger, confident, conscientiousness_big5, disgust, extraversion_big5, fear, joy, neuroticism_big5, openness_big5, sadness, tentative),List(5.5, 0.0, 6.0, 0.0, 49.0, 4.0, 2.0, 15.0, 12.5, 3.5, 48.5, 5.5, 0.0)))
(#porn,(1,List(agreeableness_big5, analytical, anger, confident, conscientiousness_big5, disgust, extraversion_big5, fear, joy, neuroticism_big5, openness_big5, sadness, tentative),List(0.5, 0.0, 16.0, 0.0, 0.5, 34.5, 49.0, 1.0, 0.0, 46.0, 16.0, 2.5, 0.0)))
...它看起来并不像教程中显示的那样:https://i0.wp.com/developer.ibm.com/clouddataservices/wp-content/uploads/sites/47/2016/01/Running-Spark-Streaming.png?zoom=2&w=980
我希望你能帮助我.What使我的图形不会每隔几秒钟就改变一次?
当我关闭应用程序和内核并尝试重新打开它们时,我会这样做:
1-重新启动内核2-再次运行所有单元
是这样的吗?
谢谢@DavidTaieb
发布于 2016-04-14 11:58:58
本教程最近由于基础服务(MessageHub、MessageConnect和Watson Tone Analyzer)中的应用程序接口更改而进行了更新。作为更改的一部分,我添加了一个StreamingListener来提供更细粒度的日志记录消息。不幸的是,我忘记更新显示正在运行的单元的屏幕截图。在任何情况下,您运行的都是最新版本,这是很棒的。
从您包含的代码片段中,看起来微批处理得到了正确的处理,并且结果成功地发送到了MessageHub/Kafka。仪表板配置为每隔几秒钟轮询一次MessageHub,但似乎需要一整分钟的时间才能刷新。我在我的测试环境中重新运行了这个应用程序,我开始注意到同样的事情。查看日志,我注意到对MessageHub的rest调用开始返回409错误代码。重新启动前端应用程序确实有助于解决这个问题。我建议你做同样的事情来解锁你。我将在未来几天进一步调查此错误,并将提供更新,一旦我有一个可接受的修复。
发布于 2016-04-14 17:33:11
Message Hub设置为,如果消费者请求在前一个请求完成之前发出,则返回409错误代码。增加轮询间隔应该可以解决此问题。
我在提供的示例中发现的另一个问题是,存在同时设置了端口9094和9093的引导服务器。截至3月3日,端口9094已关闭。请确保您仅使用端口9093。
发布于 2016-05-10 20:57:34
一小时前,当我尝试本教程时,从包含以下命令的单元格中得到以下错误:"demo.startTwitterStreaming(sc)“
注册hadoop : /tmp/BA8rP92KZiZZSkpq/jaas.conf为swift container https://kafka-rest-prod01.messagehub.services.us-south.bluemix.net:443/admin/topics https://kafka-rest-prod01.messagehub.services.us-south.bluemix.net:443/admin/topics设置JaasConfiguration配置
点击上面的链接- {"error_code":40101,"message":"Missing Key"}
Out3
名称: org.apache.hadoop.fs.swift.http.SwiftRestClient$AuthenticationPost.extractResult(SwiftRestClient.java:1293) org.apache.hadoop.fs.swift.http.SwiftRestClient$AuthenticationPost.extractResult(SwiftRestClient.java:1245)......消息:空StackTrace: null
java.lang.Thread.run(Thread.java:785)
https://stackoverflow.com/questions/36607103
复制相似问题