html - 如何使用awk根据其他字段求和字段

Question

在我的评估中，我被要求编写一个仅使用 bash 命令的 shell 脚本和另一个仅使用 SQL 查询的 shell 脚本。这些脚本应该执行以下操作: 1.清理.csv文件中的数据(暂时不重要) 2. 根据性别总结收入 3.制作一个简单的HTML表格

我已经使 SQL 查询生成了正确的数字和 HTML 文件，但在其他 bash 命令的帮助下。 对于应该只包含 bash 命令的文件，我能够得到该表，但其中一个数字是错误的。

我对 bash 脚本和 SQL 查询还很陌生，所以代码不是很优化。

以下是示例输入的简化版本: CSV input

title,site,country,year_release,box_office,director,number_of_subjects,subject,type_of_subject,race_known,subject_race,person_of_color,subject_sex,lead_actor_actress
10 Rillington Place,http://www.imdb.com/title/tt0066730/,UK,1971,-,Richard Fleischer,1,John Christie,Criminal,Unknown,,0,Male,Richard Attenborough
12 Years a Slave,http://www.imdb.com/title/tt2024544/,US/UK,2013,56700000,Steve McQueen,1, Solomon Northup,Other,Known,African American,1,Male,Chiwetel Ejiofor
127 Hours,http://www.imdb.com/title/tt1542344/,US/UK,2010,18300000,Danny Boyle,1,Aron Ralston,Athlete,Unknown,,0,Male,James Franco
1987,http://www.imdb.com/title/tt2833074/,Canada,2014,-,Ricardo Trogi,1,Ricardo Trogi,Other,Known,White,0,Male,Jean-Carl Boucher
20 Dates,http://www.imdb.com/title/tt0138987/,US,1998,537000,Myles Berkowitz,1,Myles Berkowitz,Other,Unknown,,0,Male,Myles Berkowitz
21,http://www.imdb.com/title/tt0478087/,US,2008,81200000,Robert Luketic,1,Jeff Ma,Other,Known,Asian American,1,Male,Jim Sturgess
24 Hour Party People,http://www.imdb.com/title/tt0274309/,UK,2002,1130000,Michael Winterbottom,1,Tony Wilson,Musician,Known,White,0,Male,Steve Coogan
42,http://www.imdb.com/title/tt0453562/,US,2013,95000000,Brian Helgeland,1,Jackie Robinson,Athlete,Known,African American,1,Male,Chadwick Boseman
8 Seconds,http://www.imdb.com/title/tt0109021/,US,1994,19600000,John G. Avildsen,1,Lane Frost,Athlete,Unknown,,0,Male,Luke Perry
84 Charing Cross Road,http://www.imdb.com/title/tt0090570/,US/UK,1987,1080000,David Hugh Jones,2,Frank Doel,Author,Unknown,,0,Male,Anthony Hopkins
84 Charing Cross Road,http://www.imdb.com/title/tt0090570/,US/UK,1987,1080000,David Hugh Jones,2,Helene Hanff,Author,Unknown,,0,Female,Anne Bancroft
A Beautiful Mind,http://www.imdb.com/title/tt0268978/,US,2001,171000000,Ron Howard,1,John Nash,Academic,Unknown,,0,Male,Russell Crowe
A Dangerous Method,http://www.imdb.com/title/tt1571222/,Canada/UK,2011,5700000,David Cronenberg,3,Carl Gustav Jung,Academic,Known,White,0,Male,Michael Fassbender
A Dangerous Method,http://www.imdb.com/title/tt1571222/,Canada/UK,2011,5700000,David Cronenberg,3,Sigmund Freud,Academic,Known,White,0,Male,Viggo Mortensen
A Dangerous Method,http://www.imdb.com/title/tt1571222/,Canada/UK,2011,5700000,David Cronenberg,3,Sabina Spielrein,Academic,Known,White,0,Female,Keira Knightley
A Home of Our Own,http://www.imdb.com/title/tt0107130/,US,1993,1700000,Tony Bill,1,Frances Lacey,Other,Unknown,,0,Female,Kathy Bates
A Man Called Peter,http://www.imdb.com/title/tt0048337/,US,1955,-,Henry Koster,1,Peter Marshall,Other,Known,White,0,Male,Richard Todd
A Man for All Seasons,http://www.imdb.com/title/tt0060665/,UK,1966,-,Fred Zinnemann,1,Thomas More,Historical,Known,White,0,Male,Paul Scofield
A Matador's Mistress,http://www.imdb.com/title/tt0491046/,US/UK,2008,-,Menno Meyjes,2,Lupe Sino,Actress ,Known,Hispanic (White),0,Female,PenÌÎå©lope Cruz

对于 SQL 查询，仅文件这是我目前的代码(生成正确的数字和正确的表格):

python3 csv2sqlite.py --table-name test_table --input table.csv --output table.sqlite


echo -e '<TABLE BORDER = "1">
<TR><TH>Gender</TH>
<TH>Total Amount [$]</TH>
</TR>' >> tmp1.txt
sqlite3 biopics.sqlite 'SELECT subject_sex,SUM(earnings) FROM table \
GROUP BY subject_sex;' -html > tmp2.txt
cat tmp2.txt >> tmp1.txt
echo '</TABLE>' >> tmp1.txt

cp tmp1.txt $1

cat $1

rm tmp1.txt tmp2.txt

到目前为止，对于 bash 唯一文件，这是我的代码:

echo -e '<TABLE BORDER = "1">
<TR><TH>Gender</TH>
<TH>Total Amount [$]</TH>
</TR>' >> tmp1.txt
awk -F ',' '{for (i=1;i<=NF;i++) 
        if ($1) 
            a[$13] += $5} END{for (i in a) printf("<TR><TD> %s </TD><TD> %i </TD></TR>\n", i, a[i])}' table.csv | sort | head -2 > tmp2.txt
cat tmp2.txt >> tmp1.txt
echo -e "</TABLE>" >>  tmp1.txt

cp tmp1.txt $1

cat $1

rm tmp1.txt tmp2.txt

预期的输出应该是这样的:

<TABLE BORDER = "1">
<TR><TH>Gender</TH>
<TH>Total Amount [$]</TH>
</TR>
<TR><TD>Female</TD>
<TD>8480000.0</TD>
</TR>
<TR><TD>Male</TD>
<TD>455947000.0</TD>
</TR>
</TABLE>

提前致谢!

Answer 1

#! /bin/bash

awk -F, '{
    if (NR != 1)
    {
        if (sum[$13] == "")
        {
            sum[$13]=0
        }
        sum[$13]+=$5
    }
}
END {
    print "<TABLE BORDER = \"1\">"
    print "<TR><TH>Gender</TH><TH>Total Amount [$]</TH></TR>" 
    for ( gender in sum )
    {
        print "<TR><TD>"gender"</TD>", "<TD>"sum[gender]"</TD></TR>"
    }
    print "</TABLE>"
}' table.csv

如果适合您，请在此处尝试。

更新:

我从您的评论中了解到，您希望根据总和对数据进行排序。

#! /bin/bash

awk -F, -v OFS=, '{
    if (NR != 1)
    {
        if (sum[$13] == "")
        {
            sum[$13]=0
        }
        sum[$13]+=$5
    }
}
END {
    for ( gender in sum )
    {
        print gender, sum[gender]
    }
}' table.csv | sort -nk 2,2 |

    awk -v firstline="$(sed -n '1p' table.csv)" '{
        printrow($0)
    }

    BEGIN {
        split(firstline, headers, ",")
        print "<html>"
        print "<TABLE BORDER = "1">"
        printrow(headers[5]","headers[13], 1)
    }
    END {
        print "</table>"
        print "</html>"
    }
    function printrow(row, flag) 
    {
        # if flag == 0 or null "<TD>" else "<TH>"
        len = split(row, cells, ",")
        print "<TR>"
        for (i = 1 ; i <= len ; ++i)
        {
            if (!flag)
                print "<TD>"cells[i]"</TD>"
            else
                print "<TH>"cells[i]"</TH>"
        }
        print "</TR>"
    }'

以上，我基本上把你需要的分成了2个模块，

---

操作表中的数据:

1) 整理表格

2) 根据第 2 列对数据进行排序。这个我应该在第一个 awk 脚本本身中完成，但这样会更短一些。

---

将其转换为 html 表格:

第二个 awk 脚本接收第一个脚本的输出。 它设置标题和标签。

---

我觉得这样更模块化。这只会让修改变得更容易。第一个用于数据操作的脚本，第二个用于放置标题或标签。

我个人喜欢的是为第二个 awk 脚本提供它自己的可执行文件。现在只需使用第一个脚本进行数据操作，然后将其传递给另一个脚本来设置 html 标签和标题。

可能有更好的选择，我建议了我所知道的最好的选择。