python - 使用 python 在 Apache Storm Spout 中打开文件

Question

我试图让 Apache Storm Spout 从文件中逐行读取。我曾尝试写下这些陈述，但没有成功。它给了我每次只迭代的第一行:

class SimSpout(storm.Spout):
    # Not much to do here for such a basic spout
    def initialize(self, conf, context):
        ## Open the file with read only permit
        self.f = open('data.txt', 'r')
        ## Read the first line 
        self._conf = conf
        self._context = context
        storm.logInfo("Spout instance starting...")

    # Process the next tuple
    def nextTuple(self):
        # check if it reach at the EOF to close it 
        for line in self.f.readlines():
            # Emit a random sentence
            storm.logInfo("Emiting %s" % line)
            storm.emit([line])

# Start the spout when it's invoked
SimSpout().run()

Answer 1

免责声明:由于我无法对此进行测试，因此该答案仅来自检查。

您无法保存在 initialize() 中打开的文件句柄。此编辑保存文件句柄，然后使用保存的文件句柄进行读取。它还修复(我希望)一些看起来错误的缩进。

class SimSpout(storm.Spout):
    # Not much to do here for such a basic spout
    def initialize(self, conf, context):
        ## Open the file with read only permit
        self.f = open('mydata.txt', 'r')
        self._conf = conf
        self._context = context

        storm.logInfo("Spout instance starting...")

    # Process the next tuple
    def nextTuple(self):
        # check if it reach at the EOF to close it
        for line in self.f.readlines():
            # Emit a random sentence
            storm.logInfo("Emiting %s" % line)
            storm.emit([line])

# Start the spout when it's invoked
SimSpout().run()