所以我已经阅读了所有文档并遵循了 MySQL persistence 上的教程但我仍然无法在 MySQL 数据库中保存任何类型的数据。
即使我设置了 presistence mode = row，它也不会创建任何数据库或表。
我做错了什么？
我的订阅:
python2.7 SetSubscription.py bustest4 http://localhost:5050/notify
输出:
* Asking to http://localhost:1026/v1/subscribeContext
* Headers: {'Fiware-Service': 'fiwaretestapi', 'content-type': 'application/json ', 'accept': 'application/json', 'X-Auth-Token': 'NULL'}
* Sending PAYLOAD:
{
"reference": "http://localhost:5050/notify",
"throttling": "PT5S",
"entities": [
{
"type": "",
"id": "bustest4",
"isPattern": "false"
}
],
"attributes": [
"temperature"
],
"duration": "P1M",
"notifyConditions": [
{
"condValues": [
"temperature"
],
"type": "ONCHANGE"
}
]
}
...
* Status Code: 200
{
"subscribeResponse" : {
"subscriptionId" : "5567332298add18cc3e183ac",
"duration" : "P1M",
"throttling" : "PT5S"
}
}
我的 agent_a1.conf:
#=============================================
# source configuration
# channel name where to write the notification events
cygnusagent.sources.http-source.channels = hdfs-channel mysql-channel ckan-channel
# source class, must not be changed
cygnusagent.sources.http-source.type = org.apache.flume.source.http.HTTPSource
# listening port the Flume source will use for receiving incoming notifications
cygnusagent.sources.http-source.port = 5050
# Flume handler that will parse the notifications, must not be changed
cygnusagent.sources.http-source.handler = es.tid.fiware.fiwareconnectors.cygnus.handlers.OrionRestHandler
# URL target
cygnusagent.sources.http-source.handler.notification_target = /notify
# Default service (service semantic depends on the persistence sink)
cygnusagent.sources.http-source.handler.default_service = fiwaretestapi
# Default service path (service path semantic depends on the persistence sink)
cygnusagent.sources.http-source.handler.default_service_path = /
# Number of channel re-injection retries before a Flume event is definitely discarded (-1 means infinite retries)
cygnusagent.sources.http-source.handler.events_ttl = 10
# Source interceptors, do not change
cygnusagent.sources.http-source.interceptors = ts de
# Timestamp interceptor, do not change
cygnusagent.sources.http-source.interceptors.ts.type = timestamp
# Destination extractor interceptor, do not change
cygnusagent.sources.http-source.interceptors.de.type = es.tid.fiware.fiwareconnectors.cygnus.interceptors.DestinationExtractor$Builder
# Matching table for the destination extractor interceptor, put the right absolute path to the file if necessary
# See the doc/design/interceptors document for more details
cygnusagent.sources.http-source.interceptors.de.matching_table = /usr/cygnus/conf/matching_table.conf
# ============================================
# OrionMySQLSink configuration
# channel name from where to read notification events
cygnusagent.sinks.mysql-sink.channel = mysql-channel
# sink class, must not be changed
cygnusagent.sinks.mysql-sink.type = es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionMySQLSink
# the FQDN/IP address where the MySQL server runs
cygnusagent.sinks.mysql-sink.mysql_host = localhost
# the port where the MySQL server listes for incomming connections
cygnusagent.sinks.mysql-sink.mysql_port = 3306
# a valid user in the MySQL server
cygnusagent.sinks.mysql-sink.mysql_username = root
# password for the user above
cygnusagent.sinks.mysql-sink.mysql_password = ***********
# how the attributes are stored, either per row either per column (row, column)
cygnusagent.sinks.mysql-sink.attr_persistence = row
#=============================================
# mysql-channel configuration
# channel type (must not be changed)
cygnusagent.channels.mysql-channel.type = memory
# capacity of the channel
cygnusagent.channels.mysql-channel.capacity = 1000
# amount of bytes that can be sent per transaction
cygnusagent.channels.mysql-channel.transactionCapacity = 100
我的 cygnus_instance_c1.conf:
# Who to run cygnus as. Note that you may need to use root if you want
# to run cygnus in a privileged port (<1024)
CYGNUS_USER=root
# Where is the config folder
CONFIG_FOLDER=/usr/cygnus/conf
# Which is the config file
CONFIG_FILE=/usr/cygnus/conf/agent_a1.conf
# Name of the agent. The name of the agent is not trivial, since it is the base for the Flume parameters
# naming conventions, e.g. it appears in .sources.http-source.channels=...
AGENT_NAME=cygnususer
# Name of the logfile located at /var/log/cygnus. It is important to put the extension '.log' in order to the log rotation works properly
LOGFILE_NAME=cygnus.log
# Administration port. Must be unique per instance
ADMIN_PORT=8081
我的 cygnus.log:
Info: Sourcing environment configuration script /usr/cygnus/conf/flume-env.sh
Warning: JAVA_HOME is not set!
+ exec /usr/bin/java -Xmx20m -Dflume.log.file=cygnus.log -cp '/usr/cygnus/conf:/usr/cygnus/lib/*:/usr/cygnus/plugins.d/cygnus/lib/*:/usr/cygnus/plugins.d/cygnus/libext/*' -Djava.library.path= es.tid.fiware.fiwareconnectors.cygnus.nodes.CygnusApplication -p 8081 -f /usr/cygnus/conf/agent_a1.conf -n root
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/cygnus/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/cygnus/plugins.d/cygnus/lib/cygnus-0.7.1-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
编辑:
因此，在进行一些更改后，我让日志文件开始工作，但我发现 8081 端口已被使用。您能解释一下 ADMIN_PORT 的用途吗？建议使用哪个端口？
日志文件:
02 Jun 2015 05:16:40,680 INFO [conf-file-poller-0] (org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run:133) - Reloading configuration file:/usr/cygnus/conf/agent_a1.conf
02 Jun 2015 05:16:40,685 INFO [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1016) - Processing:mysql-sink
02 Jun 2015 05:16:40,686 INFO [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1016) - Processing:mysql-sink
02 Jun 2015 05:16:40,686 INFO [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:930) - Added sinks: mysql-sink Agent: cygnusagent
02 Jun 2015 05:16:40,686 INFO [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1016) - Processing:mysql-sink
02 Jun 2015 05:16:40,687 INFO [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1016) - Processing:mysql-sink
02 Jun 2015 05:16:40,687 INFO [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1016) - Processing:mysql-sink
02 Jun 2015 05:16:40,687 INFO [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1016) - Processing:mysql-sink
02 Jun 2015 05:16:40,687 INFO [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1016) - Processing:mysql-sink
02 Jun 2015 05:16:40,692 INFO [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration.validateConfiguration:140) - Post-validation flume configuration contains configuration for agents: [cygnusagent]
02 Jun 2015 05:16:40,692 WARN [conf-file-poller-0] (org.apache.flume.node.AbstractConfigurationProvider.getConfiguration:138) - No configuration found for this host:root
02 Jun 2015 05:16:40,693 INFO [conf-file-poller-0] (org.apache.flume.node.Application.stopAllComponents:101) - Shutting down configuration: { sourceRunners:{} sinkRunners:{} channels:{} }
02 Jun 2015 05:16:40,693 INFO [conf-file-poller-0] (org.apache.flume.node.Application.startAllComponents:138) - Starting new configuration:{ sourceRunners:{} sinkRunners:{} channels:{} }
02 Jun 2015 05:16:40,694 INFO [conf-file-poller-0] (es.tid.fiware.fiwareconnectors.cygnus.nodes.CygnusApplication.startManagementInterface:85) - Starting a Jetty server listening on port 8081 (Management Interface)
02 Jun 2015 05:16:40,695 INFO [conf-file-poller-0] (org.apache.flume.node.Application.stopAllComponents:101) - Shutting down configuration: { sourceRunners:{} sinkRunners:{} channels:{} }
02 Jun 2015 05:16:40,695 INFO [conf-file-poller-0] (org.apache.flume.node.Application.startAllComponents:138) - Starting new configuration:{ sourceRunners:{} sinkRunners:{} channels:{} }
02 Jun 2015 05:16:40,695 INFO [conf-file-poller-0] (es.tid.fiware.fiwareconnectors.cygnus.nodes.CygnusApplication.startManagementInterface:85) - Starting a Jetty server listening on port 8081 (Management Interface)
02 Jun 2015 05:16:40,696 INFO [Thread-26] (org.mortbay.log.Slf4jLog.info:67) - jetty-6.1.26
02 Jun 2015 05:16:40,704 WARN [Thread-26] (org.mortbay.log.Slf4jLog.warn:76) - failed SocketConnector@0.0.0.0:8081: java.net.BindException: Address already in use
02 Jun 2015 05:16:40,704 WARN [Thread-26] (org.mortbay.log.Slf4jLog.warn:76) - failed Server@4f1b95f: java.net.BindException: Address already in use
02 Jun 2015 05:16:40,704 FATAL [Thread-26] (es.tid.fiware.fiwareconnectors.cygnus.http.JettyServer.run:63) - Fatal error running the Management Interface. Details=Address already in use
02 Jun 2015 05:16:40,705 INFO [Thread-27] (org.mortbay.log.Slf4jLog.info:67) - jetty-6.1.26
02 Jun 2015 05:16:40,709 WARN [Thread-27] (org.mortbay.log.Slf4jLog.warn:76) - failed SocketConnector@0.0.0.0:8081: java.net.BindException: Address already in use
02 Jun 2015 05:16:40,709 WARN [Thread-27] (org.mortbay.log.Slf4jLog.warn:76) - failed Server@ed4c222: java.net.BindException: Address already in use
02 Jun 2015 05:16:40,709 FATAL [Thread-27] (es.tid.fiware.fiwareconnectors.cygnus.http.JettyServer.run:63) - Fatal error running the Management Interface. Details=Address already in use
编辑 2:
我在 Context Broker 上更新实体的脚本:
BASE_URL = 'http://localhost:1026'
UPDATE_URL = BASE_URL+'/ngsi10/updateContext'
HEADERS = {
'Content-Type': 'application/json',
'Accept': 'application/json',
'Fiware-Service' : 'fiwaretestapi',
'Fiware-ServicePath': '/'
}
UPDATE_EXAMPLE = {
"contextElements": [
{
"type": "",
"isPattern": "false",
"id": "bustest4",
"attributes": [
{
"name": "temperature",
"type": "int",
"value": "99"
}
]
}
],
"updateAction": "APPEND"
}
def post(url, data):
""""""
req = urllib2.Request(url, data, HEADERS)
f = urllib2.urlopen(req)
result = json.loads(f.read())
f.close()
return result
if __name__ == "__main__":
print post(UPDATE_URL, json.dumps(UPDATE_EXAMPLE))
编辑 3:
即使我在 cygnus 代理配置上将管理端口设置为 8085，它也会尝试绑定(bind)到 8081，这正常吗？
这是来自 cygnus 的日志:
time=2015-06-12T05:56:39.820EDT | lvl=INFO | trans= | function=start | comp=Cygnu s | msg=org.apache.flume.instrumentation.MonitoredCounterGroup[94] : Component ty pe: CHANNEL, name: mysql-channel started
time=2015-06-12T05:56:39.821EDT | lvl=INFO | trans= | function=startAllComponents | comp=Cygnus | msg=org.apache.flume.node.Application[173] : Starting Sink mysql -sink
time=2015-06-12T05:56:39.821EDT | lvl=INFO | trans= | function=startAllComponents | comp=Cygnus | msg=org.apache.flume.node.Application[184] : Starting Source htt p-source
time=2015-06-12T05:56:39.821EDT | lvl=INFO | trans= | function=startManagementInt erface | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.nodes.CygnusAppl ication[85] : Starting a Jetty server listening on port 8081 (Management Interfac e)
time=2015-06-12T05:56:39.822EDT | lvl=INFO | trans= | function=start | comp=Cygnu s | msg=es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionMySQLSink[151] : [mysql- sink] Startup completed
time=2015-06-12T05:56:39.823EDT | lvl=INFO | trans= | function=info | comp=Cygnus | msg=org.mortbay.log.Slf4jLog[67] : jetty-6.1.26
time=2015-06-12T05:56:39.824EDT | lvl=INFO | trans= | function=info | comp=Cygnus | msg=org.mortbay.log.Slf4jLog[67] : jetty-6.1.26
time=2015-06-12T05:56:39.825EDT | lvl=INFO | trans= | function=info | comp=Cygnus | msg=org.mortbay.log.Slf4jLog[67] : Started SocketConnector@0.0.0.0:5050
time=2015-06-12T05:56:39.825EDT | lvl=INFO | trans= | function=start | comp=Cygnu s | msg=org.apache.flume.instrumentation.MonitoredCounterGroup[94] : Component ty pe: SOURCE, name: http-source started
time=2015-06-12T05:56:39.827EDT | lvl=WARN | trans= | function=warn | comp=Cygnus | msg=org.mortbay.log.Slf4jLog[76] : failed SocketConnector@0.0.0.0:8081: java.n et.BindException: Address already in use
time=2015-06-12T05:56:39.827EDT | lvl=WARN | trans= | function=warn | comp=Cygnus | msg=org.mortbay.log.Slf4jLog[76] : failed Server@1c9c5521: java.net.BindExcept ion: Address already in use
time=2015-06-12T05:56:39.827EDT | lvl=FATAL | trans= | function=run | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.http.JettyServer[63] : Fatal error r unning the Management Interface. Details=Address already in use
订阅时记录:
time=2015-06-12T06:03:56.529EDT | lvl=INFO | trans=1434103313-190-0000000000 | fu nction=getEvents | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.handle rs.OrionRestHandler[153] : Starting transaction (1434103313-190-0000000000)
time=2015-06-12T06:03:56.535EDT | lvl=INFO | trans=1434103313-190-0000000000 | fu nction=getEvents | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.handle rs.OrionRestHandler[239] : Received data ({ "subscriptionId" : "557aae8c98add18c c3e183b6", "originator" : "localhost", "contextResponses" : [ { "contex tElement" : { "type" : "thing", "isPattern" : "false", "id" : "autocarro1", "attributes" : [ { "name" : "temperatu re", "type" : "int", "value" : "95", "metadatas" : [ { "name" : "TimeInstant", "type" : "ISO8601", "value" : "2015-06-03T09:17:44.046583Z" } ] } ] }, "statusCode" : { "code" : "200", "reasonPhrase" : "OK" } } ]})
time=2015-06-12T06:03:56.540EDT | lvl=INFO | trans=1434103313-190-0000000000 | fu nction=getEvents | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.handle rs.OrionRestHandler[261] : Event put in the channel (id=1983722072, ttl=10)
time=2015-06-12T06:03:56.724EDT | lvl=INFO | trans=1434103313-190-0000000000 | fu nction=process | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.sinks.Or ionSink[126] : Event got from the channel (id=1983722072, headers={timestamp=1434 103436542, content-type=application/json, transactionId=1434103313-190-0000000000 , fiware-service=fiwaretestapi, fiware-servicepath=, ttl=10, destination=autocarr o1_thing}, bodyLength=657)
time=2015-06-12T06:03:57.260EDT | lvl=INFO | trans=1434103313-190-0000000000 | fu nction=persist | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.sinks.Or ionMySQLSink[227] : [mysql-sink] Persisting data at OrionMySQLSink. Database: fiw aretestapi, Table: autocarro1_thing, Data: 1434103436,2015-06-12T06:03:56.542,aut ocarro1,thing,temperature,thing,95,[{"name":"TimeInstant","type":"ISO8601","value ":"2015-06-03T09:17:44.046583Z"}]
time=2015-06-12T06:03:57.270EDT | lvl=INFO | trans=1434103313-190-0000000000 | fu nction=process | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.sinks.Or ionSink[187] : Finishing transaction (1434103313-190-0000000000)
最佳答案
正如我在日志中看到的，很可能 Cygnus 正在运行但没有启动任何 Flume 组件(任何源、 channel 或接收器)。这是由于一些配置错误。
关于 agent_a1.conf 文件:
- 它缺少源、 channel 和汇的列表:
cygnusagent.sources = http-源 cygnusagent.sinks = mysql 接收器 cygnusagent.channels = mysql channel cygnusagent.sources.http-source.channels值应该是mysql-channel
关于cygnus_instance_c1.conf:
AGENT_NAME值必须是cygnusagent
你安装的是哪个版本？您是将 Cygnus 作为服务还是作为独立进程重新运行？
此外，您可以尝试以 DEBUG 模式启动 Cygnus 吗？只需编辑 /usr/cygnus/conf/log4j.properties 文件即可。
执行建议的更改并查看日志如何演变! :)
编辑 1
这样的“ fatal error ”并没有那么致命。这是 Cygnus 0.7.1 中出现的错误，目前已修复。无论如何，即使在 0.7.1 中，它也不会影响 Cygnus 的正常行为，因为管理端口仅用于检索版本，并不重要。
您是否尝试向 Orion 发送一些更新上下文以便 Cygnus 收到通知？甚至通过模拟通知？请参阅 Cygnus Quick Start Guide有关如何进行此类模拟的示例。
编辑 2
Cygnus 包名称 es.tid.fiware.fiwareconnectors.cygnus... 从 0.8.0 版开始被替换为 com.telefonica.iot.cygnus... (或者可能是 0.9.0)。
关于mysql - Cygnus 不保留 MySql 数据库中的数据，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/30511082/