Using FreeSWITCH with MRCP

Here are links to relevant FreeSWITCH information for interfacing with MRCP:

• mod_unimrcp - Allows FreeSWITCH to connect to an MRCP server for ASR and TTS. Supports both MRCPv1 and v2.
• mod_dptools: play_and_detect_speech - allows you to play a question prompt (e.g. via TTS) and at the same time start speech recognition, e.g. via an ASR recognizer provided by mod_unimrcp
  
  • Can be invoked from Lua script, for example:
    

    session:execute("set", "tts_engine=unimrcp:voicegain1-mrcp2");
    session:execute("set", "tts_voice=Catherine");
    session:execute("play_and_detect_speech", 
    "say:What is your phone number detect:unimrcp {start-input-timers=false,define-grammar=true,no-input-timeout=5000}builtin:grammar/phone")
    
    asrResult = session:getVariable("detect_speech_result");

    • Note, you do not have to use Voicegain TTS. You can use pre-recorded prompts instead.
    • The example uses built-in grammars. To learn more abut built-in grammars included with Voicegain see here.
    • GRXML grammars will also work. For samples of yes/no, digit, and zip code grammars see here.
  • Result of the recognition can be retrieved from this variable: detect_speech_result
    The variable contains the result in XML format (NLSML).
  • You can also use this command "play_and_detect_speech_close_asr" to release ASR MRCP session immediately after recognition. The normal command "play_and_detect_speech" holds onto ASR MRCP session until the end of the call.

Setting up mod_unimrcp for Voicegain

First you need to make sure that it gets built. In build/modules.conf.in uncomment this line: #asr_tts/mod_unimrcp

After the build, before starting FreeSWITCH you will need to:

• Add <load module="mod_unimrcp"/> to autoload_configs/modules.conf.xml
  (you can put it in <!-- ASR /TTS --> section because that is where it logically belongs)
• Modify content of autoload_config/unimrcp.conf.xml
  We assume that you will want to use both ASR and TTS via Voicegain MRCP, so you will need to point both default-asr-profile and default-tts-profile to the voicegain1-mrcp2 profile you will create in mrcp_profiles folder.

Here is an example MRCP v2 profile for connecting to Voicegain MRCP
(you can put it in file voicegain-mrcp-v2.xml):

<include>
  <!-- Voicegain MRCP Server 1 MRCPv2 -->
  <profile name="voicegain1-mrcp2" version="2">
    <!--param name="client-ext-ip" value="auto"-->
    <param name="client-ip" value="auto"/>
    <param name="client-port" value="5090"/>
    <!-- set to the external IP on which e.g. Voicegain MRCP Proxy is deployed -->
    <param name="server-ip" value="10.5.5.152"/>
    <param name="server-port" value="8060"/>
    <!--param name="force-destination" value="1"/-->
    <param name="sip-transport" value="tcp"/>
    <!--param name="ua-name" value="FreeSWITCH"/-->
    <!--param name="sdp-origin" value="FreeSWITCH"/-->
    <!--param name="rtp-ext-ip" value="auto"/-->
    <param name="rtp-ip" value="auto"/>
    <param name="rtp-port-min" value="4000"/>
    <param name="rtp-port-max" value="5000"/>
    <!--param name="playout-delay" value="50"/-->
    <!--param name="max-playout-delay" value="200"/-->
    <!--param name="ptime" value="20"/-->
    <param name="codecs" value="PCMU PCMA L16/96/8000"/>
    <param name="jsgf-mime-type" value="application/jsgf"/>

    <!-- Add any default MRCP params for SPEAK requests here -->
    <synthparams>
    </synthparams>

    <!-- Add any default MRCP params for RECOGNIZE requests here -->
    <recogparams>
      <!--param name="start-input-timers" value="false"/-->
    </recogparams>
  </profile>
</include>

NOTE 1: it is important that the port range used by the Unimrcp Client:
<param name="rtp-port-min" value="4000"/>
<param name="rtp-port-max" value="5000"/>

 is accessible from outside FreeSWITCH, otherwise, the TTS via MRCP will not work. Also, these ports may not overlap with the ports used by FreeSWITCH.
BTW, current TTS over our MRCP is very primitive. Let us know if you need better quality TTS - we can make the Google TTS accessible.

NOTE 2: in some setups the auto values of :
<param name="client-ip" value="auto"/>
and
<param name="rtp-ip" value="auto"/>
may not work and you will have to manually specify the external IP.