linux - 格式化 Masscan 输出

Question

Masscan 的 -oG 选项有以下输出:

# Masscan 1.0.6 scan initiated Mon May  6 08:45:19 2019
# Ports scanned: TCP(13107;1-13107) UDP(0;) SCTP(0;) PROTOCOLS(0;)
Host: 192.168.1.1 ()  Ports: 8000/open/tcp//unknown//
Host: 192.168.1.2 ()  Ports: 3478/open/tcp//unknown//
Host: 192.168.1.3 ()   Ports: 8000/open/tcp//unknown//
Host: 192.168.1.1 ()        Ports: 80/open/tcp//http//
Host: 192.168.1.2 ()        Ports: 443/open/tcp//https//
Host: 192.168.1.4 () Ports: 443/open/tcp//https//
Host: 192.168.1.3 () Ports: 80/open/tcp//http//
Host: 192.168.1.4 () Ports: 80/open/tcp//http//

如何使用 awk、cut、grep、sed 等操作此输出以获得以下格式:

192.168.1.1 80,8000
192.168.1.2 443,3478
192.168.1.3 80,8000
192.168.1.4 80,443

Answer 1

试试这个:

#!/bin/bash

# define testcontent
content=$(cat << EOT
# Masscan 1.0.6 scan initiated Mon May  6 08:45:19 2019
# Ports scanned: TCP(13107;1-13107) UDP(0;) SCTP(0;) PROTOCOLS(0;)
Host: 192.168.1.1 ()  Ports: 8000/open/tcp//unknown//
Host: 192.168.1.2 ()  Ports: 3478/open/tcp//unknown//
Host: 192.168.1.3 ()   Ports: 8000/open/tcp//unknown//
Host: 192.168.1.1 ()        Ports: 80/open/tcp//http//
Host: 192.168.1.2 ()        Ports: 443/open/tcp//https//
Host: 192.168.1.4 () Ports: 443/open/tcp//https//
Host: 192.168.1.3 () Ports: 80/open/tcp//http//
Host: 192.168.1.4 () Ports: 80/open/tcp//http//
EOT
)

# declare associative array
declare -A dict 

# loop over all ip lines
while read -r ip port; do
   # save ports
   dict[$ip]+="$port "
         # ignore lines start with #, grep ip an port from content 
done < <(sed '/^#/d;s/Host: \([^ ]*\).*Ports: \([0-9]*\).*/\1 \2/' <<< "$content") 

# loop over assocative array
for key in  "${!dict[@]}"; do

   # sort ports in string
   sorted=$(echo "${dict[$key]}" | tr " " "\n" | sort -n | tr "\n" ,)

   # extract leading ,
   ports="${sorted#*,}"

   # print key an ports without tailing ,
   printf "%s %s\n" "$key" "${ports%,*}"
done | sort  

输出

192.168.1.1 80,8000
192.168.1.2 443,3478
192.168.1.3 80,8000
192.168.1.4 80,443