java - 使用 jfreechart 绘制音频信号(幅度与时间)

Question

我继承了一个代码片段，它绘制了给定文件的音频波形。但是这个波形是使用 JAVA vector 图形构建的简单图像，没有任何标签、轴信息等。我想将它移植到 jfreechart 以增加它的信息值(value)。我的问题是代码至少可以说是神秘的。

public class Plotter {
AudioInputStream audioInputStream;
Vector<Line2D.Double> lines = new Vector<Line2D.Double>();
String errStr;
Capture capture = new Capture();
double duration, seconds;
//File file;
String fileName = "out.png";
SamplingGraph samplingGraph;
String waveformFilename;
Color imageBackgroundColor = new Color(20,20,20);

public Plotter(URL url, String waveformFilename) throws Exception {
    if (url != null) {
        try {
            errStr = null;
            this.fileName = waveformFilename;
            audioInputStream = AudioSystem.getAudioInputStream(url);
            long milliseconds = (long)((audioInputStream.getFrameLength() * 1000) / audioInputStream.getFormat().getFrameRate());
            duration = milliseconds / 1000.0;
            samplingGraph = new SamplingGraph();
            samplingGraph.createWaveForm(null);     

        } catch (Exception ex) { 
            reportStatus(ex.toString());
            throw ex;
        }
    } else {
        reportStatus("Audio file required.");
    }
}
/**
 * Render a WaveForm.
 */
class SamplingGraph implements Runnable {

    private Thread thread;
    private Font font10 = new Font("serif", Font.PLAIN, 10);
    private Font font12 = new Font("serif", Font.PLAIN, 12);
    Color jfcBlue = new Color(000, 000, 255);
    Color pink = new Color(255, 175, 175);


    public SamplingGraph() {
    }


    public void createWaveForm(byte[] audioBytes) {

        lines.removeAllElements();  // clear the old vector

        AudioFormat format = audioInputStream.getFormat();
        if (audioBytes == null) {
            try {
                audioBytes = new byte[
                    (int) (audioInputStream.getFrameLength() 
                    * format.getFrameSize())];
                audioInputStream.read(audioBytes);
            } catch (Exception ex) { 
                reportStatus(ex.getMessage());
                return; 
            }
        }
        int w = 500;
        int h = 200;
        int[] audioData = null;
        if (format.getSampleSizeInBits() == 16) {
             int nlengthInSamples = audioBytes.length / 2;
             audioData = new int[nlengthInSamples];
             if (format.isBigEndian()) {
                for (int i = 0; i < nlengthInSamples; i++) {
                     /* First byte is MSB (high order) */
                     int MSB = (int) audioBytes[2*i];
                     /* Second byte is LSB (low order) */
                     int LSB = (int) audioBytes[2*i+1];
                     audioData[i] = MSB << 8 | (255 & LSB);
                 }
             } else {
                 for (int i = 0; i < nlengthInSamples; i++) {
                     /* First byte is LSB (low order) */
                     int LSB = (int) audioBytes[2*i];
                     /* Second byte is MSB (high order) */
                     int MSB = (int) audioBytes[2*i+1];
                     audioData[i] = MSB << 8 | (255 & LSB);
                 }
             }
         } else if (format.getSampleSizeInBits() == 8) {
             int nlengthInSamples = audioBytes.length;
             audioData = new int[nlengthInSamples];
             if (format.getEncoding().toString().startsWith("PCM_SIGN")) {
                 for (int i = 0; i < audioBytes.length; i++) {
                     audioData[i] = audioBytes[i];
                 }
             } else {
                 for (int i = 0; i < audioBytes.length; i++) {
                     audioData[i] = audioBytes[i] - 128;
                 }
             }
        }

        int frames_per_pixel = audioBytes.length / format.getFrameSize()/w;
        byte my_byte = 0;
        double y_last = 0;
        int numChannels = format.getChannels();
        for (double x = 0; x < w && audioData != null; x++) {
            int idx = (int) (frames_per_pixel * numChannels * x);
            if (format.getSampleSizeInBits() == 8) {
                 my_byte = (byte) audioData[idx];
            } else {
                 my_byte = (byte) (128 * audioData[idx] / 32768 );
            }
            double y_new = (double) (h * (128 - my_byte) / 256);
            lines.add(new Line2D.Double(x, y_last, x, y_new));
            y_last = y_new;
        }
        saveToFile();
    }


    public void saveToFile() {            
        int w = 500;
        int h = 200;
        int INFOPAD = 15;

        BufferedImage bufferedImage = new BufferedImage(w, h, BufferedImage.TYPE_INT_RGB);
        Graphics2D g2 = bufferedImage.createGraphics();

        createSampleOnGraphicsContext(w, h, INFOPAD, g2);            
        g2.dispose();
        // Write generated image to a file
        try {
            // Save as PNG
            File file = new File(fileName);
            System.out.println(file.getAbsolutePath());
            ImageIO.write(bufferedImage, "png", file);
            JOptionPane.showMessageDialog(null, 
                    new JLabel(new ImageIcon(fileName)));
        } catch (IOException e) {
        }
    }


    private void createSampleOnGraphicsContext(int w, int h, int INFOPAD, Graphics2D g2) {            
        g2.setBackground(imageBackgroundColor);
        g2.clearRect(0, 0, w, h);
        g2.setColor(Color.white);
        g2.fillRect(0, h-INFOPAD, w, INFOPAD);

        if (errStr != null) {
            g2.setColor(jfcBlue);
            g2.setFont(new Font("serif", Font.BOLD, 18));
            g2.drawString("ERROR", 5, 20);
            AttributedString as = new AttributedString(errStr);
            as.addAttribute(TextAttribute.FONT, font12, 0, errStr.length());
            AttributedCharacterIterator aci = as.getIterator();
            FontRenderContext frc = g2.getFontRenderContext();
            LineBreakMeasurer lbm = new LineBreakMeasurer(aci, frc);
            float x = 5, y = 25;
            lbm.setPosition(0);
            while (lbm.getPosition() < errStr.length()) {
                TextLayout tl = lbm.nextLayout(w-x-5);
                if (!tl.isLeftToRight()) {
                    x = w - tl.getAdvance();
                }
                tl.draw(g2, x, y += tl.getAscent());
                y += tl.getDescent() + tl.getLeading();
            }
        } else if (capture.thread != null) {
            g2.setColor(Color.black);
            g2.setFont(font12);
            //g2.drawString("Length: " + String.valueOf(seconds), 3, h-4);
        } else {
            g2.setColor(Color.black);
            g2.setFont(font12);
            //g2.drawString("File: " + fileName + "  Length: " + String.valueOf(duration) + "  Position: " + String.valueOf(seconds), 3, h-4);

            if (audioInputStream != null) {
                // .. render sampling graph ..
                g2.setColor(jfcBlue);
                for (int i = 1; i < lines.size(); i++) {
                    g2.draw((Line2D) lines.get(i));
                }

                // .. draw current position ..
                if (seconds != 0) {
                    double loc = seconds/duration*w;
                    g2.setColor(pink);
                    g2.setStroke(new BasicStroke(3));
                    g2.draw(new Line2D.Double(loc, 0, loc, h-INFOPAD-2));
                }
            }
        }
    }

    public void start() {
        thread = new Thread(this);
        thread.setName("SamplingGraph");
        thread.start();
        seconds = 0;
    }

    public void stop() {
        if (thread != null) {
            thread.interrupt();
        }
        thread = null;
    }

    public void run() {
        seconds = 0;
        while (thread != null) {
            if ( (capture.line != null) && (capture.line.isActive()) ) {
                long milliseconds = (long)(capture.line.getMicrosecondPosition() / 1000);
                seconds =  milliseconds / 1000.0;
            }
            try { thread.sleep(100); } catch (Exception e) { break; }                              
            while ((capture.line != null && !capture.line.isActive())) 
            {
                try { thread.sleep(10); } catch (Exception e) { break; }
            }
        }
        seconds = 0;
    }
} // End class SamplingGraph

/** 
 * Reads data from the input channel and writes to the output stream
 */
class Capture implements Runnable {

    TargetDataLine line;
    Thread thread;

    public void start() {
        errStr = null;
        thread = new Thread(this);
        thread.setName("Capture");
        thread.start();
    }

    public void stop() {
        thread = null;
    }

    private void shutDown(String message) {
        if ((errStr = message) != null && thread != null) {
            thread = null;
            samplingGraph.stop();                
            System.err.println(errStr);
        }
    }

    public void run() {

        duration = 0;
        audioInputStream = null;

        // define the required attributes for our line, 
        // and make sure a compatible line is supported.

        AudioFormat format = audioInputStream.getFormat();
        DataLine.Info info = new DataLine.Info(TargetDataLine.class, 
            format);

        if (!AudioSystem.isLineSupported(info)) {
            shutDown("Line matching " + info + " not supported.");
            return;
        }

        // get and open the target data line for capture.

        try {
            line = (TargetDataLine) AudioSystem.getLine(info);
            line.open(format, line.getBufferSize());
        } catch (LineUnavailableException ex) { 
            shutDown("Unable to open the line: " + ex);
            return;
        } catch (SecurityException ex) { 
            shutDown(ex.toString());
            //JavaSound.showInfoDialog();
            return;
        } catch (Exception ex) { 
            shutDown(ex.toString());
            return;
        }

        // play back the captured audio data
        ByteArrayOutputStream out = new ByteArrayOutputStream();
        int frameSizeInBytes = format.getFrameSize();
        int bufferLengthInFrames = line.getBufferSize() / 8;
        int bufferLengthInBytes = bufferLengthInFrames * frameSizeInBytes;
        byte[] data = new byte[bufferLengthInBytes];
        int numBytesRead;

        line.start();

        while (thread != null) {
            if((numBytesRead = line.read(data, 0, bufferLengthInBytes)) == -1) {
                break;
            }
            out.write(data, 0, numBytesRead);
        }

        // we reached the end of the stream.  stop and close the line.
        line.stop();
        line.close();
        line = null;

        // stop and close the output stream
        try {
            out.flush();
            out.close();
        } catch (IOException ex) {
            ex.printStackTrace();
        }

        // load bytes into the audio input stream for playback

        byte audioBytes[] = out.toByteArray();
        ByteArrayInputStream bais = new ByteArrayInputStream(audioBytes);
        audioInputStream = new AudioInputStream(bais, format, audioBytes.length / frameSizeInBytes);

        long milliseconds = (long)((audioInputStream.getFrameLength() * 1000) / format.getFrameRate());
        duration = milliseconds / 1000.0;

        try {
            audioInputStream.reset();
        } catch (Exception ex) { 
            ex.printStackTrace(); 
            return;
        }

        samplingGraph.createWaveForm(audioBytes);
    }
} // End class Capture    

我已经看过好几次了，知道下面的部分是计算音频值的地方，但我的问题是我不知道如何在那个时候检索时间信息，即那个值属于什么时间间隔。

 int frames_per_pixel = audioBytes.length / format.getFrameSize()/w;
            byte my_byte = 0;
            double y_last = 0;
            int numChannels = format.getChannels();
            for (double x = 0; x < w && audioData != null; x++) {
                int idx = (int) (frames_per_pixel * numChannels * x);
                if (format.getSampleSizeInBits() == 8) {
                     my_byte = (byte) audioData[idx];
                } else {
                     my_byte = (byte) (128 * audioData[idx] / 32768 );
                }
                double y_new = (double) (h * (128 - my_byte) / 256);
                lines.add(new Line2D.Double(x, y_last, x, y_new));
                y_last = y_new;
            }

我想使用 jfreechart 的 XYSeriesPLot 绘制它，但在计算所需的 x(time) 和 y 值时遇到问题(这是振幅，但在此代码中是 y_new)？

我知道这是一件很容易的事情，但我对整个音频东西都是新手，我了解音频文件背后的理论，但这似乎是一个简单的问题却很难解决

enter link description here

Answer 1

要认识到的关键是，在提供的代码中，预计情节的分辨率要比实际音频数据低得多。例如，考虑以下波形:

绘图代码然后将数据表示为图中的蓝色框:

当框为 1 像素宽时，这对应于端点为 (x,y_last) 和 (x,y_new) 的线。如您所见，当波形足够平滑时，从 y_last 到 y_new 的振幅范围非常接近框内的样本。

现在，当尝试以逐像素方式(光栅显示)渲染波形时，这种表示会很方便。但是，对于 XYPlot 图(可以在 jfreechart 中找到)，您只需要指定一系列 (x,y) 点，XYPlot 会负责绘制这些点之间的线段。这对应于下图中的绿线:

理论上，您可以按原样 将每个样本提供给 XYPlot。但是，除非您的样本很少，否则绘制起来往往会很繁重。因此，通常人们会先对数据进行下采样。如果波形足够平滑，则下采样过程会减少为抽取(即每 N 个样本取 1 个)。抽取因子 N 然后控制渲染性能和波形近似精度之间的权衡。请注意，如果在提供的代码中使用抽取因子 frames_per_pixel 来生成良好的光栅显示(即，您希望看到的波形特征没有被 block 状像素外观隐藏，并且不显示混叠伪影)，同样的因素对于 XYPlot 应该仍然足够(事实上，您可以减少采样更多)。

就将样本映射到时间/振幅轴而言，我不会使用在提供的绘图代码中定义的 x 和 y 参数:它们只是适用于光栅类型显示的像素索引(如上面的蓝框表示)。

我会通过除以采样率(您可以从 format.getFrameRate() 获得)将样本索引(提供的代码中的 idx)直接映射到时间轴)。 同样，我通过将 audioData[idx] 样本除以 8 位的 128，将全量程样本值映射到 [-1,+1] 范围-per-sample 数据，对于 16-bits-per-sample 数据为 32768。

w 和 h 参数的主要目的仍然是配置绘图区域大小，但不再直接需要计算 XYPlot 输入(XYPlot 本身负责将时间/振幅值映射到像素坐标)。另一方面，w 参数还用于确定要绘制的点数的额外目的。现在您可能想要根据波形可以承受多少抽取而不会显示太多失真来控制点数，或者您可以保持原样以最大可用绘图分辨率显示波形(有一些性能成本)。 但是请注意，如果您希望显示少于 w 个样本的波形，您可能必须将 frames_per_pixel 转换为浮点值。