C# Mel-Spectrogram 梅尔频谱

目录

介绍

Main features

Philosophy of NWaves

效果 

项目

代码

下载

C# Mel-Spectrogram 梅尔频谱

介绍

利用NWaves实现Mel-Spectrogram 梅尔频谱

NWaves github 地址：https://github.com/ar1st0crat/NWaves

NWaves is a .NET DSP library with a lot of audio processing functions.

Main features

•  major DSP transforms (FFT, DCT, MDCT, STFT, FWT, Hilbert, Hartley, Mellin, cepstral, Goertzel)
•  signal builders (sine, white/pink/red/Perlin noise, awgn, triangle, sawtooth, square, pulse, ramp, ADSR, wavetable)
•  basic LTI digital filters (moving average, comb, Savitzky-Golay, pre/de-emphasis, DC removal, RASTA)
•  FIR/IIR filtering (offline and online), zero-phase filtering
•  BiQuad filters (low-pass, high-pass, band-pass, notch, all-pass, peaking, shelving)
•  1-pole filters (low-pass, high-pass)
•  IIR filters (Bessel, Butterworth, Chebyshev I & II, Elliptic, Thiran)
•  basic operations (convolution, cross-correlation, rectification, amplification, fade / crossfade)
•  block convolution (overlap-add / overlap-save offline and online)
•  basic filter design & analysis (group delay, zeros/poles, BP, BR, HP from/to LP, SOS, combining filters)
•  state space representation of LTI filters
•  FIR filter design: frequency sampling, window-sinc, equiripple (Remez / Parks-McClellan)
•  IIR filter design: IirNotch / IirPeak / IirCombNotch / IirCombPeak
•  non-linear filters (median filter, distortion effects, bit crusher)
•  windowing functions (Hamming, Blackman, Hann, Gaussian, Kaiser, KBD, triangular, Lanczos, flat-top, Bartlett)
•  periodograms (Welch / Lomb-Scargle)
•  psychoacoustic filter banks (Mel, Bark, Critical Bands, ERB, octaves) and VTLN warping
•  customizable feature extraction (time-domain, spectral, MFCC, PNCC/SPNCC, LPC, LPCC, PLP, AMS)
•  preconfigured MFCC extractors: HTK (MFCC-FB24), Slaney (MFCC-FB40)
•  LPC conversions: LPC<->cepstrum, LPC<->LSF
•  feature post-processing (mean and variance normalization, adding deltas) and CSV serialization
•  spectral features (centroid, spread, flatness, entropy, rolloff, contrast, crest, decrease, noiseness, MPEG7)
•  harmonic features (harmonic centroid and spread, inharmonicity, tristimulus, odd-to-even ratio)
•  time-domain characteristics (rms, energy, zero-crossing rate, entropy)
•  pitch tracking (autocorrelation, YIN, ZCR + Schmitt trigger, HSS/HPS, cepstrum)
•  chromagram (chroma feature extractor)
•  time scale modification (phase vocoder, PV with identity phase locking, WSOLA, PaulStretch)
•  simple resampling, interpolation, decimation
•  bandlimited resampling
•  wavelets: haar, db, symlet, coiflet
•  polyphase filters
•  noise reduction (spectral subtraction, sciPy-style Wiener filtering)
•  sound effects (echo, tremolo, wahwah, phaser, chorus, vibrato, flanger, pitch shift, morphing, robotize, whisperize)
•  3D/Stereo audio (stereo panning, stereo and ping-pong delay, ITD-ILD, binaural panning)
•  envelope following
•  dynamics processing (limiter / compressor / expander / noise gate)
•  harmonic/percussive separation
•  Griffin-Lim algorithm
•  Karplus-Strong synthesis
•  PADSynth synthesis
•  adaptive filtering (LMS, NLMS, LMF, SignLMS, RLS)
•  simple modulation/demodulation (AM, ring, FM, PM)
•  simple audio playback and recording

Philosophy of NWaves

NWaves was initially intended for research, visualizing and teaching basics of DSP and sound programming.

Usually, DSP code is quite complicated and difficult to read, because it's full of optimizations (which is actually a very good thing). NWaves project aims in particular at achieving a tradeoff between good understandable code/design and satisfactory performance. Yet, the main purpose of this lib is to offer the DSP codebase that would be:

• easy to read and understand
• easy to incorporate into existing projects
• easy to port to other programming languages and frameworks
• even possibly treated as the DSP/audio textbook.

效果 

项目

代码

using NWaves.Audio;
using NWaves.FeatureExtractors;
using NWaves.FeatureExtractors.Options;
using NWaves.Filters.Fda;
using NWaves.Signals;
using NWaves.Windows;
using System;
using System.IO;
using System.Text;
using System.Windows.Forms;

namespace C__Mel_Spectrogram_梅尔频谱
{
    public partial class Form1 : Form
    {
        public Form1()
        {
            InitializeComponent();
        }

        WaveFile waveContainer;
        StringBuilder sb = new StringBuilder();

        private void button1_Click(object sender, EventArgs e)
        {
            using (var stream = new FileStream("你好.wav", FileMode.Open))
            {
                waveContainer = new WaveFile(stream);
            }

            DiscreteSignal left = waveContainer[Channels.Left];
            //DiscreteSignal right = waveContainer[Channels.Right];

            //Mel-Spectrogram
            var fftSize = 1024;
            var hopSize = 512;
            var melCount = 16;
            int samplingRate = left.SamplingRate;

            var mfccExtractor = new FilterbankExtractor(
               new FilterbankOptions
               {
                   SamplingRate = samplingRate,
                   FrameSize = fftSize,
                   FftSize = fftSize,
                   HopSize = hopSize,
                   Window = WindowType.Hann,
                   FilterBank = FilterBanks.Triangular(fftSize, samplingRate, FilterBanks.MelBands(melCount, samplingRate)),
                   // if power = 1.0
                   // SpectrumType = SpectrumType.Magnitude
               });

            var mfccVectors = mfccExtractor.ParallelComputeFrom(left);

            sb.Clear();

            foreach (var item in mfccVectors)
            {
                sb.AppendLine(String.Join(",", item));
            }

            textBox1.Text = sb.ToString();
        }
    }
}

using NWaves.Audio;
using NWaves.FeatureExtractors;
using NWaves.FeatureExtractors.Options;
using NWaves.Filters.Fda;
using NWaves.Signals;
using NWaves.Windows;
using System;
using System.IO;
using System.Text;
using System.Windows.Forms;

namespace C__Mel_Spectrogram_梅尔频谱
{
    public partial class Form1 : Form
    {
        public Form1()
        {
            InitializeComponent();
        }

        WaveFile waveContainer;
        StringBuilder sb = new StringBuilder();

        private void button1_Click(object sender, EventArgs e)
        {
            using (var stream = new FileStream("你好.wav", FileMode.Open))
            {
                waveContainer = new WaveFile(stream);
            }

            DiscreteSignal left = waveContainer[Channels.Left];
            //DiscreteSignal right = waveContainer[Channels.Right];

            //Mel-Spectrogram
            var fftSize = 1024;
            var hopSize = 512;
            var melCount = 16;
            int samplingRate = left.SamplingRate;

            var mfccExtractor = new FilterbankExtractor(
               new FilterbankOptions
               {
                   SamplingRate = samplingRate,
                   FrameSize = fftSize,
                   FftSize = fftSize,
                   HopSize = hopSize,
                   Window = WindowType.Hann,
                   FilterBank = FilterBanks.Triangular(fftSize, samplingRate, FilterBanks.MelBands(melCount, samplingRate)),
                   // if power = 1.0
                   // SpectrumType = SpectrumType.Magnitude
               });


            var mfccVectors = mfccExtractor.ParallelComputeFrom(left);

            sb.Clear();

            foreach (var item in mfccVectors)
            {
                sb.AppendLine(String.Join(",", item));
            }

            textBox1.Text = sb.ToString();
        }
    }
}

下载

源码下载