首页
学习
活动
专区
圈层
工具
发布
社区首页 >问答首页 >未能从voltdb出口到kafka

未能从voltdb出口到kafka
EN

Stack Overflow用户
提问于 2015-05-14 03:19:16
回答 1查看 259关注 0票数 0

VoltDB是企业5.1.2

Kafka为2.9.2-0.8.1.1,也尝试了2.10-0.8.2.1。

VoltDB为192.168.56.101

卡夫卡电话: 192.168.56.102

下面是我对VoltDB的部署配置:

代码语言:javascript
复制
<deployment>
  <cluster hostcount="1" sitesperhost="4" kfactor="0" />

  <commandlog enabled="true">
    <frequency time="400" transactions="1000"  />
  </commandlog>

  <export>
    <configuration enabled="true" type="kafka" stream="archive">
      <property name="metadata.broker.list">192.168.56.102:9092</property>
      <property name="producer.type">sync</property>
      <property name="batch.mode">true</property>
    </configuration>
  </export>
</deployment>

模式被定义为:

代码语言:javascript
复制
drop table person if exists;
create table person (
  ic varchar(9) not null,
  first_name varchar(20) not null,
  last_name varchar(20) not null,
  middle_name varchar(20),
  gender tinyint not null,
  dob timestamp not null,
  date_created timestamp default now
);
partition table person on column ic;

export table person to stream archive;

对于卡夫卡的server.properties,我只添加了这一行

代码语言:javascript
复制
auto.create.topics.enable=true

我第一次创办卡夫卡是这样的:

代码语言:javascript
复制
bin/zookeeper-server-startsh config/zookeeper.properties
bin/kafka-server-start.sh config/server.properties

启动VoltDB时,会遇到以下异常:

代码语言:javascript
复制
david@u14voltdb:~$ voltdb create catalog.jar --deployment=config.xml
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=64m; support was removed in 8.0
Initializing VoltDB...

 _    __      ____  ____  ____ 
| |  / /___  / / /_/ __ \/ __ )
| | / / __ \/ / __/ / / / __  |
| |/ / /_/ / / /_/ /_/ / /_/ / 
|___/\____/_/\__/_____/_____/

--------------------------------

Build: 5.1.2 voltdb-5.1.2-0-g6d05c33-local Enterprise Edition
Connecting to VoltDB cluster as the leader...
Host id of this node is: 0
Starting VoltDB with trial license. License expires on May 31, 2015.
Initializing the database and command logs. This may take a moment...
WARN: Failed to send producer request with correlation id 2 to broker 0     with data for partitions [voltdbexportPERSON,0]
java.nio.channels.ClosedChannelException
    at kafka.network.BlockingChannel.send(BlockingChannel.scala:97)
    at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:72)
    at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:71)
    at kafka.producer.SyncProducer$$anonfun$send$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(SyncProducer.scala:102)
    at kafka.producer.SyncProducer$$anonfun$send$1$$anonfun$apply$mcV$sp$1.apply(SyncProducer.scala:102)
    at kafka.producer.SyncProducer$$anonfun$send$1$$anonfun$apply$mcV$sp$1.apply(SyncProducer.scala:102)
    at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33)
    at kafka.producer.SyncProducer$$anonfun$send$1.apply$mcV$sp(SyncProducer.scala:101)
    at kafka.producer.SyncProducer$$anonfun$send$1.apply(SyncProducer.scala:101)
    at kafka.producer.SyncProducer$$anonfun$send$1.apply(SyncProducer.scala:101)
    at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33)
    at kafka.producer.SyncProducer.send(SyncProducer.scala:100)
    at kafka.producer.async.DefaultEventHandler.kafka$producer$async$DefaultEventHandler$$send(DefaultEventHandler.scala:255)
    at kafka.producer.async.DefaultEventHandler$$anonfun$dispatchSerializedData$1.apply(DefaultEventHandler.scala:106)
    at kafka.producer.async.DefaultEventHandler$$anonfun$dispatchSerializedData$1.apply(DefaultEventHandler.scala:100)
    at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:80)
    at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:80)
    at scala.collection.Iterator$class.foreach(Iterator.scala:631)
    at scala.collection.mutable.HashTable$$anon$1.foreach(HashTable.scala:161)
    at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:194)
    at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:39)
    at scala.collection.mutable.HashMap.foreach(HashMap.scala:80)
    at kafka.producer.async.DefaultEventHandler.dispatchSerializedData(DefaultEventHandler.scala:100)
    at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:72)
    at kafka.producer.Producer.send(Producer.scala:76)
    at kafka.javaapi.producer.Producer.send(Producer.scala:42)
    at org.voltdb.exportclient.KafkaExportClient$KafkaExportDecoder.onBlockCompletion(KafkaExportClient.java:217)
    at org.voltdb.export.processors.GuestProcessor$2.run(GuestProcessor.java:223)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at org.voltcore.utils.CoreUtils$7$1.run(CoreUtils.java:735)
    at java.lang.Thread.run(Thread.java:745)

在卡夫卡那边,我一直收到这样的信息:

代码语言:javascript
复制
[2015-05-14 00:40:08,197] INFO Closing socket connection to /192.168.56.101. (kafka.network.Processor)

有什么建议吗?

EN

回答 1

Stack Overflow用户

回答已采纳

发布于 2015-05-14 05:53:17

这纯粹是一个与卡夫卡背景有关的问题。在该设置中,有一个注释设置:

代码语言:javascript
复制
advertised.host.name=something

只需将“某事”替换为运行Kafka的服务器的IP地址即可。这在Kafka - Unable to send a message to a remote server using Java找到。

票数 0
EN
页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接:

https://stackoverflow.com/questions/30228664

复制
相关文章

相似问题

领券
问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档