javascript - GroupBy CSV 文件中的数组元素并有助于减少代码

Question

导入的每个 csv 文件都具有相同的数据结构。 我需要按每个“[服务类型”]求和[“净费用金额”]。 我目前正在通过将每个唯一的['服务类型']分配给它们自己的数组来做到这一点。我当前的脚本可能有点矫枉过正，但很容易遵循，但是我正在寻找一种更紧凑的方法来执行此操作，否则该脚本可能会变得很长。

const fs = require('fs')
const { parse } = require('csv-parse')

// Arrays for each service type
const GroundShipments = []
const HomeDeliveryShipments = []
const SmartPostShipments = []
const Shipments = []

[Shipments] 数组将保存所有数据，我假设这是数组 我们希望与您合作

//functions for each service type
function isGround(shipment) {
 return shipment['Service Type'] === 'Ground'
}
   
function isHomeDelivery(data) {
 return data['Service Type'] === 'Home Delivery'
}
function isSmartpost(shipment) {
    return shipment['Service Type'] === 'SmartPost'
}
function isShipment(shipment) {
    return shipment['Service Type'] === 'Ground' || shipment['Service Type'] === 'Home Delivery' ||
    shipment['Service Type'] === 'SmartPost'
}
// Import csv file / perform business rules by service type
// output sum total by each service type
fs.createReadStream('repco.csv')
    .pipe(parse({
        columns: true
    }))
    .on('data', (data) => {
        //push data to proper service type array
        // Ground
         if (isGround(data)) {
            GroundShipments.push(data)
            }
        // Home Delivery
        if (isHomeDelivery(data)) {
            HomeDeliveryShipments.push(data)
            }
        // Smartpost
        if (isSmartpost(data)) {
            SmartPostShipments.push(data)
            }
        // All shipment types, including Ground, Home Delivery, and Smartpost
        if (isShipment(data)) {
            Shipments.push(data)
        }    
    })
    .on('error', (err) => {
        console.log(err)
    })
    .on('end', (data) => {
        // sum data by service type
        // Ground Only
        const sumGround = GroundShipments.reduce((acc, data) =>
        acc + parseFloat(data['Net Charge Amount']), 0)
        // Home Delivery Only
        const sumHomeDelivery = HomeDeliveryShipments.reduce((acc, data) =>
        acc + parseFloat(data['Net Charge Amount']), 0)
        // SmartPost Only
        const sumSmartPost = SmartPostShipments.reduce((acc, data) =>
        acc + parseFloat(data['Net Charge Amount']), 0)
        // All services
        const sumAllShipments = Shipments.reduce((acc, data) => 
        acc + parseFloat(data['Net Charge Amount']), 0)
      
        //output sum by service type to console
        console.log(`${GroundShipments.length} Ground shipments: ${sumGround}`)
        console.log(`${HomeDeliveryShipments.length} Home Delivery shipments: ${sumHomeDelivery}`)
        console.log(`${SmartPostShipments.length} Smartpost shipments: ${sumSmartPost}`)
        console.log(`${Shipments.length} All shipments: ${sumAllShipments}`)
    })

这是控制台输出: [1]:/image/FltTU.png

我希望用一个数组[Shipments]来输出每个唯一的['服务类型']和[的总和，而不是用它自己的数组和函数来分隔每个['服务类型'] ‘净收费金额’]

Answer 1

简化此操作的两个关键是:

1. 将 CSV 解析与数据处理分开
2. 使用groupBy函数

首先，您应该将 CSV 解析为简单的 JS 数组。然后您可以使用常规的 JS 实用函数来操作数据，例如 groupBy 函数。它是一个实用程序，可以在 lodash 和 ramda 库中找到。它可能会作为 .group 方法添加到 vanilla JS 中，但这还需要一段时间。

我正在寻找一个示例问题来使用我自己的 JS 评估框架，所以我在那里回答了你的问题:

您可以自己探索底层的val:https://www.val.town/stevekrouse.exampleGroupByShppingCSV

关于我的答案，有几件事在正常的 NodeJS 代码库中没有意义，但我必须做才能使其在 val.town 中工作(异步/等待，使用自定义 groupBy 方法而不是导入一)。如果您需要帮助使其在您的应用程序中运行，请告诉我。