streamline audio feature extraction -pg电子麻将胡了
streamline audio feature extraction
since r2019b
description
audiofeatureextractor encapsulates multiple audio feature
extractors into a streamlined and modular implementation.
creation
description
afe = audiofeatureextractor()
afe = audiofeatureextractor(name=value)afe using one or more name-value
arguments.
properties
object functions
| extract audio features | |
| set nondefault parameter values for individual feature extractors | |
| output mapping and individual feature extractor parameters | |
| create matlab function compatible with c/c code generation | |
| plot extracted audio features |
examples
extract multiple audio features
read in an audio signal.
[audioin,fs] = audioread("counting-16-44p1-mono-15secs.wav");create an audiofeatureextractor object that extracts the mfcc, delta mfcc, delta-delta mfcc, pitch, spectral centroid, zero-crossing rate, and short-time energy of the signal. use a 30 ms analysis window with 20 ms overlap.
afe = audiofeatureextractor( ... samplerate=fs, ... window=hamming(round(0.03*fs),"periodic"), ... overlaplength=round(0.02*fs), ... mfcc=true, ... mfccdelta=true, ... mfccdeltadelta=true, ... pitch=true, ... spectralcentroid=true, ... zerocrossrate=true, ... shorttimeenergy=true);
call extract to extract the audio features from the audio signal.
features = extract(afe,audioin);
use info to determine which column of the feature extraction matrix corresponds to the requested pitch extraction.
idx = info(afe)
idx = struct with fields:
mfcc: [1 2 3 4 5 6 7 8 9 10 11 12 13]
mfccdelta: [14 15 16 17 18 19 20 21 22 23 24 25 26]
mfccdeltadelta: [27 28 29 30 31 32 33 34 35 36 37 38 39]
spectralcentroid: 40
pitch: 41
zerocrossrate: 42
shorttimeenergy: 43
plot the detected pitch over time.
t = linspace(0,size(audioin,1)/fs,size(features,1)); plot(t,features(:,idx.pitch)) title("pitch") xlabel("time (s)") ylabel("frequency (hz)")
plot the zero-crossing rate over time.
plot(t,features(:,idx.zerocrossrate)) title("zero-crossing rate") xlabel("time (s)")
plot the short-time energy over time.
plot(t,features(:,idx.shorttimeenergy)) title("short-time energy") xlabel("time (s)")
extract features from dataset
create an audio datastore that points to audio samples included with audio toolbox®.
folder = fullfile(matlabroot,"toolbox","audio","samples"); ads = audiodatastore(folder);
find all files that correspond to a sample rate of 44.1 khz and then the datastore.
keepfile = cellfun(@(x)contains(x,"44p1"),ads.files);
ads = subset(ads,keepfile);convert the data to a array. tall arrays are evaluated only when you request them explicitly using . matlab® automatically optimizes the queued calculations by minimizing the number of passes through the data. if you have parallel computing toolbox™, you can spread the calculations across multiple workers. the audio data is represented as an m-by-1 tall cell array, where m is the number of files in the audio datastore.
adstall = tall(ads)
starting parallel pool (parpool) using the 'local' profile ...
connected to the parallel pool (number of workers: 6).
adstall =
m×1 tall cell array
{ 539648×1 double}
{ 227497×1 double}
{ 8000×1 double}
{ 685056×1 double}
{ 882688×2 double}
{1115760×2 double}
{ 505200×2 double}
{3195904×2 double}
: :
: :
create an audiofeatureextractor object to extract the mel spectrum, bark spectrum, erb spectrum, and linear spectrum from each audio file. use the default analysis window and overlap length for the spectrum extraction.
afe = audiofeatureextractor(samplerate=44.1e3, ... melspectrum=true, ... barkspectrum=true, ... erbspectrum=true, ... linearspectrum=true);
define a function so that audio features are extracted from each cell of the tall array. call to evaluate the tall array.
specstall = cellfun(@(x)extract(afe,x),adstall,uniformoutput=false); specs = gather(specstall);
evaluating tall expression using the parallel pool 'local': - pass 1 of 1: completed in 14 sec evaluation completed in 14 sec
the specs variable returned from gather is a numfiles-by-1 cell array, where numfiles is the number of files in the datastore. each element of the cell array is a numhops-by-numfeatures-by-numchannels array, where the number of hops and number of channels depends on the length and number of channels of the audio file, and the number of features is the requested number of features from the audio data.
numfiles = numel(specs)
numfiles = 12
[numhops1,numfeaturesfile1,numchanelsfile1] = size(specs{1})numhops1 = 1053
numfeaturesfile1 = 620
numchanelsfile1 = 1
[numhops2,numfeaturesfile2,numchanelsfile2] = size(specs{2})numhops2 = 443
numfeaturesfile2 = 620
numchanelsfile2 = 1
visualize extracted audio features
use plotfeatures to visualize audio features extracted with an audiofeatureextractor object.
read in an audio signal from a file.
[audioin,fs] = audioread("counting-16-44p1-mono-15secs.wav");create an audiofeatureextractor object that extracts the gammatone cepstral coefficients (gtccs) and the delta of the gtccs. set the samplerate property to the sample rate of the audio signal, and use the default values for the other properties.
afe = audiofeatureextractor(samplerate=fs,gtcc=true,gtccdelta=true);
plot the features extracted from the audio signal.
plotfeatures(afe,audioin)
algorithms
the audiofeatureextractor creates a feature extraction pipeline based on
your selected features. to reduce computations, audiofeatureextractor reuses
intermediary representations and outputs some intermediate representations as features.
for example, to create an object that extracts the centroid of the bark spectrum, the flux
of the bark spectrum, the pitch, the harmonic ratio, and the delta-delta of the mfcc, specify
the audiofeatureextractor as
follows.
afe = audiofeatureextractor( ... spectraldescriptorinput="barkspectrum", ... spectralcentroid=true, ... spectralflux=true, ... pitch=true, ... harmonicratio=true, ... mfccdeltadelta=true)
afe =
audiofeatureextractor with properties:
properties
window: [1024×1 double]
overlaplength: 512
samplerate: 44100
fftlength: []
spectraldescriptorinput: 'barkspectrum'
enabled features
mfccdeltadelta, spectralcentroid, spectralflux, pitch, harmonicratio
disabled features
linearspectrum, melspectrum, barkspectrum, erbspectrum, mfcc, mfccdelta
gtcc, gtccdelta, gtccdeltadelta, spectralcrest, spectraldecrease, spectralentropy
spectralflatness, spectralkurtosis, spectralrolloffpoint, spectralskewness, spectralslope, spectralspread
to extract a feature, set the corresponding property to true.
for example, obj.mfcc = true, adds mfcc to the list of enabled features.
note
because audiofeatureextractor reuses intermediary representations, the
features output from audiofeatureextractor might not correspond with the
default configuration of features output by corresponding individual feature
extractors.
extended capabilities
version history
introduced in r2019bsee also
| audiodatastore | audiodataaugmenter | |
select a web site
choose a web site to get translated content where available and see local events and offers. based on your location, we recommend that you select: .
you can also select a web site from the following list: