Netstream Get Audio Only (SoundMixer.computeSpectrum)

Question

Netstream As sound only?

I'm trying to put the sound coming from a netstream into a Sound variable so I can visualize it like in this tutorial. Adobe ActionScript 3.0 * Accessing raw sound data

Problem is, search results only find how to attach a video to a video object, and not a sound object.

private function handleAccept(e:MouseEvent):void 
    {
    myNS.attachAudio(Microphone.getEnhancedMicrophone(0));
    myNS.publish("audio");

    var s:Sound = new Sound(theirNS.play("audio"));//??
    s.play(); 

    //Custom tranformation ahead..
    }

Answer 1

To be able to use SoundMixer.computeSpectrum() with the audio of an RTMP stream, you should start by enabling the access to the audio raw data in the server side like this :

For Adobe / Flash Media Server ( AMS / FMS ) :

Application.xml :

<Application> 

    <!-- ... -->

    <Client>
        <Access>
            <AudioSampleAccess enabled="true">/</AudioSampleAccess>
        </Access>
    </Client>

    <!-- ... -->

</Application>

or using Main.asc :

application.onConnect = function( p_client, p_autoSenseBW )
{
    // ...

    p_client.audioSampleAccess = "/";

    // ...      
}

---

For Wowza Media Server :

Application.xml :

<Application>

    <!-- ... -->

    <Client>
        <Access>
            <StreamAudioSampleAccess>*</StreamAudioSampleAccess>
        </Access>
    </Client>

    <!-- ... -->

</Application>

---

For RED5 :

red5-web.xml :

<bean id="rtmpSampleAccess" class="org.red5.server.stream.RtmpSampleAccess">
    <property name="audioAllowed" value="true"/>
</bean> 

---

Then you have just to play your stream as you did usually and for the graphic representation of the sound wave data, you can use, for example, the example code available in the documentation page of SoundMixer.computeSpectrum() :

var server:String = 'rtmp://127.0.0.1/vod', stream:String = 'sample',
    nc:NetConnection, ns:NetStream, vd:Video,
    spectrum:Sprite, last_status:String = '';

nc = new NetConnection();
nc.connect(server);
nc.addEventListener(NetStatusEvent.NET_STATUS, on_NetStatus);
nc.addEventListener(AsyncErrorEvent.ASYNC_ERROR, function(e:AsyncErrorEvent): void {});

function on_NetStatus(e:NetStatusEvent):void
{
    var code:String = e.info.code;
    switch (code)
    {
        case 'NetConnection.Connect.Success' :

            ns = new NetStream(nc);
            ns.bufferTime = 3;
            ns.addEventListener(NetStatusEvent.NET_STATUS, on_NetStatus);
            ns.addEventListener(AsyncErrorEvent.ASYNC_ERROR, function(e:AsyncErrorEvent): void {});

            vd = new Video(320,180);
            vd.x = vd.y = 0;
            vd.attachNetStream(ns);
            addChild(vd);

            spectrum = new Sprite();
            spectrum.x = spectrum.y = 0;
            addChild(spectrum);

            ns.play(stream);

            break;

        case 'NetStream.Play.Start' :
            addEventListener(Event.ENTER_FRAME, on_EnterFrame);
            break;

        case 'NetStream.Buffer.Flush':          
            if(last_status == 'NetStream.Play.Stop'){
                removeEventListener(Event.ENTER_FRAME, on_EnterFrame);
            }
            break;
    }

    last_status = code;

}

function on_EnterFrame(event:Event):void
{
    var bytes:ByteArray = new ByteArray();
    const PLOT_HEIGHT:int = stage.stageHeight / 2;
    const CHANNEL_LENGTH:int = 256;

    SoundMixer.computeSpectrum(bytes, false, 0);

    var g:Graphics = spectrum.graphics;

    g.clear();

    g.lineStyle(0, 0xffffff);
    g.beginFill(0x00ff00);
    g.moveTo(0, PLOT_HEIGHT);

    var n:Number = 0;

    for (var i:int = 0; i < CHANNEL_LENGTH; i++)
    {
        n = (bytes.readFloat() * PLOT_HEIGHT);
        g.lineTo(i * 2, PLOT_HEIGHT - n);
    }

    g.lineTo(CHANNEL_LENGTH * 2, PLOT_HEIGHT);
    g.endFill();

    g.lineStyle(0, 0xCC0066);
    g.beginFill(0xCC0066, 0.5);
    g.moveTo(CHANNEL_LENGTH * 2, PLOT_HEIGHT);

    for (i = CHANNEL_LENGTH; i > 0; i--)
    {
        n = (bytes.readFloat() * PLOT_HEIGHT);
        g.lineTo(i * 2, PLOT_HEIGHT - n);
    }

    g.lineTo(0, PLOT_HEIGHT);
    g.endFill();
}

sample here is an FLV video file provided with AMS / FMS. You can of course use any kind of supported video or audio files ( MP4, MP3, ... ).

The above code gives you something like this :

Hope that can help.

Answer 2

That example uses SoundMixer.computeSpectrum and thus as long as the NetStream you are playing is not an RTMP stream, everything will work fine.

this method cannot be used to extract data from RTMP streams

Assuming you are not using a RTMP stream, here is an example using NetStream vs Sound that I have that overlays the waveform on top on the video that is playing:

package {
    import flash.display.Graphics;
    import flash.events.Event;
    import flash.media.SoundMixer;
    import flash.display.Sprite;
    import flash.net.NetStream;
    import flash.net.NetConnection;
    import flash.utils.ByteArray;
    import flash.media.Video;

    public class Main extends Sprite {

        const PLOT_HEIGHT:int = 200;
        const CHANNEL_LENGTH:int = 256;
        private var bytes:ByteArray = new ByteArray();

        private var videoURL:String = "YourVideoURL";
        private var connection:NetConnection;
        private var video:Video = new Video();
        private var waveForm:Sprite = new Sprite();

        public function Main() {

            connection = new NetConnection();
            connection.connect(null);
            video.width = parent.stage.stageWidth;
            video.height = parent.stage.stageHeight;
            addChild(video);

            var stream:NetStream = new NetStream(connection);
            stream.client = new CustomClient();
            video.attachNetStream(stream);
            stream.play(videoURL);

            addChild(waveForm);
            addEventListener(Event.ENTER_FRAME, onEnterFrame);
        }

        function onEnterFrame(event:Event):void {
            SoundMixer.computeSpectrum(bytes, false, 0);

            var g:Graphics = waveForm.graphics;

            g.clear();
            g.lineStyle(0, 0xffeeff);
            g.beginFill(0xeeffee);
            g.moveTo(0, PLOT_HEIGHT);

            var n:Number = 0;

            // left channel
            for (var i:int = 0; i < CHANNEL_LENGTH; i++) {
                n = (bytes.readFloat() * PLOT_HEIGHT);
                g.lineTo(i * 2, PLOT_HEIGHT - n);
            }
            g.lineTo(CHANNEL_LENGTH * 2, PLOT_HEIGHT);
            g.endFill();

            // right channel
            g.lineStyle(0, 0xeeffee);
            g.beginFill(0xffeeff, 0.5);
            g.moveTo(CHANNEL_LENGTH * 2, PLOT_HEIGHT);

            for (i = CHANNEL_LENGTH; i > 0; i--) {
                n = (bytes.readFloat() * PLOT_HEIGHT);
                g.lineTo(i * 2, PLOT_HEIGHT - n);
            }
            g.lineTo(0, PLOT_HEIGHT);
            g.endFill();
        }

        function onPlaybackComplete(event:Event) {
            removeEventListener(Event.ENTER_FRAME, onEnterFrame);
        }

    }
}