Implementing AI Hearing using C++

March 26, 2016
Οrfeas
AI Programming, Unreal Engine 4 C++ Tutorials
20 Comments

In this post we’re going to create an AI Character that hears our player footsteps and follows him. If you are interested in creating a “Seeing” sense for your AI, check out this post. Before we start, here is the end result:

To achieve the above functionality, we’re going to need a couple of things:

A playable character with a PawnNoiseEmitter component, which will be used to “report” into the game that a sound has been played.
A Controller for our AI character
A custom AI Character which will contain a reference for our Behavior Tree and a PawnSensing component
A Behavior Tree for our AI Logic
A Blackboard to store values for our Behavior Tree
A footstep sound (Optional)

While reading this list you will notice that the footsteps sound is optional, this is because in UE4 we explicitly tell the game that we’ve played a sound somewhere with a certain volume. The engine itself doesn’t care if we have actually played a sound or not. In case you want to include the same footstep sound I’ve used in the video above, here is the download link. In order to get started, create a Third Person C++ Template project.

Before we even begin to write any code, right when your template project loads up, add a nav mesh bounds volume to cover your whole level!

Setting up our playable Character

To set up our character, open up it’s header file, and include the following library right before the .generated.h file:

#include "Perception/PawnSensingComponent.h"

Then, add the following declarations:

public:
    /*The function that is going to play the sound and report it to our game*/
	UFUNCTION(BlueprintCallable, Category = AI)
	void ReportNoise(USoundBase* SoundToPlay,float Volume);

	/*A Pawn Noise Emitter component which is used in order to emit the sounds to nearby AIs*/
	UPROPERTY(VisibleAnywhere,BlueprintReadWrite)
	UPawnNoiseEmitterComponent* PawnNoiseEmitterComp;

public:

/*The function that is going to play the sound and report it to our game*/

UFUNCTION(BlueprintCallable, Category = AI)

void ReportNoise(USoundBase* SoundToPlay,float Volume);

/*A Pawn Noise Emitter component which is used in order to emit the sounds to nearby AIs*/

UPROPERTY(VisibleAnywhere,BlueprintReadWrite)

UPawnNoiseEmitterComponent* PawnNoiseEmitterComp;

Then, switch to your character’s source file and inside your constructor, type in the following initialization:

PawnNoiseEmitterComp = CreateDefaultSubobject<UPawnNoiseEmitterComponent>(TEXT("PawnNoiseEmitterComp"));

1	PawnNoiseEmitterComp = CreateDefaultSubobject<UPawnNoiseEmitterComponent>(TEXT("PawnNoiseEmitterComp"));

Moreover, type in the following implementation of the ReportNoise function:

void ATPBlogProjectCharacter::ReportNoise(USoundBase* SoundToPlay,float Volume)
{
	//If we have a valid sound to play, play the sound and
	//report it to our game
	if (SoundToPlay)
	{
		//Play the actual sound
		UGameplayStatics::PlaySoundAtLocation(GetWorld(), SoundToPlay, GetActorLocation(), Volume);

		//Report that we've played a sound with a certain volume in a specific location
		MakeNoise(Volume, this, GetActorLocation());
	}
	
}

void ATPBlogProjectCharacter::ReportNoise(USoundBase* SoundToPlay,float Volume)

{

//If we have a valid sound to play, play the sound and

//report it to our game

if (SoundToPlay)

{

//Play the actual sound

UGameplayStatics::PlaySoundAtLocation(GetWorld(), SoundToPlay, GetActorLocation(), Volume);

//Report that we've played a sound with a certain volume in a specific location

MakeNoise(Volume, this, GetActorLocation());

}

Note here we’re doing two things. We first play the given sound and then we’re using the built-in function MakeNoise to inform our game that we have played a sound.

In case you copy and pasted the code above don’t forget to edit my Character’s class name to match yours.

Now that we’re done with our character’s C++ logic and what we need to tell our character to play a sound when he touches the floor. To do that we’re going to use Animation Notifies. Think of Animation Notifies like functions, which get called in a specific frame of an animation that we decide. For more information about them, check out the official UE4 documentation.

Locate your character’s run animation and add a new Notify (I named my notify GenerateFootstepSound) when your character’s feet touch the ground. I’ve placed my notifies in frames 8 and 18 like the following image suggests:

anim_notify — Click on the image to enlarge in a new tab

Then, go to the Event Graph of your animation blueprint and add in the following logic for the Notifies we’ve created above:

footstep_generation — Click on the image to enlarge in a new tab

Notice that I imported the sound I’ve mentioned above and I have hardcoded its reference for the given function. It would be more flexible to create a separate USoundBase* property inside the Character’s header file and assign values through its Blueprint. However, in this case, this is adequate.

Compile and save your Blueprint. If you play with your Character right now, you will hear the generated sounds for each footstep! Let’s move on!

Creating our AI Character

Add a new C++ class which is based on the Character class and name it MyAICharacter. Inside it’s header file, right before the .generated.h add the following includes:

#include "Perception/PawnSensingComponent.h"
#include "BehaviorTree/BehaviorTree.h"

1 2	#include "Perception/PawnSensingComponent.h" #include "BehaviorTree/BehaviorTree.h"

Then, type in the following declarations:

public:
    /*A Pawn Sensing Component, responsible for sensing other Pawns*/
	UPROPERTY(VisibleAnywhere)
	UPawnSensingComponent* PawnSensingComp;

	/*Hearing function - will be executed when we hear a Pawn*/
	UFUNCTION()
	void OnHearNoise(APawn* PawnInstigator, const FVector& Location, float Volume);

	/*A Behavior Tree reference*/
	UPROPERTY(EditDefaultsOnly)
	UBehaviorTree* BehaviorTree;

public:

/*A Pawn Sensing Component, responsible for sensing other Pawns*/

UPROPERTY(VisibleAnywhere)

UPawnSensingComponent* PawnSensingComp;

/*Hearing function - will be executed when we hear a Pawn*/

UFUNCTION()

void OnHearNoise(APawn* PawnInstigator, const FVector& Location, float Volume);

/*A Behavior Tree reference*/

UPROPERTY(EditDefaultsOnly)

UBehaviorTree* BehaviorTree;

Then, switch to your source file, and type in the following code:

// Sets default values
AMyAICharacter::AMyAICharacter()
{
 	// Set this character to call Tick() every frame.  You can turn this off to improve performance if you don't need it.
	PrimaryActorTick.bCanEverTick = true;

	//Initializing our Pawn Sensing comp and our behavior tree reference
	PawnSensingComp = CreateDefaultSubobject<UPawnSensingComponent>(TEXT("PawnSensingComp"));
	BehaviorTree = CreateDefaultSubobject<UBehaviorTree>(TEXT("BehaviorTreeReference"));
}

// Called when the game starts or when spawned
void AMyAICharacter::BeginPlay()
{
	Super::BeginPlay();

	if (PawnSensingComp)
	{
		//Registering the delegate which will fire when we hear something
		PawnSensingComp->OnHearNoise.AddDynamic(this, &AMyAICharacter::OnHearNoise);
	}
	
}

void AMyAICharacter::OnHearNoise(APawn* PawnInstigator, const FVector& Location, float Volume)
{

	//AMyAIController* Con = Cast<AMyAIController>(GetController());
	
	//We don't want to hear ourselves
	if (Con && PawnInstigator != this)
	{
		//Updates our target based on what we've heard.
		//Con->SetSensedTarget(PawnInstigator);
	}
}

// Sets default values

AMyAICharacter::AMyAICharacter()

{

// Set this character to call Tick() every frame. You can turn this off to improve performance if you don't need it.

PrimaryActorTick.bCanEverTick = true;

//Initializing our Pawn Sensing comp and our behavior tree reference

PawnSensingComp = CreateDefaultSubobject<UPawnSensingComponent>(TEXT("PawnSensingComp"));

BehaviorTree = CreateDefaultSubobject<UBehaviorTree>(TEXT("BehaviorTreeReference"));

}

// Called when the game starts or when spawned

void AMyAICharacter::BeginPlay()

{

Super::BeginPlay();

if (PawnSensingComp)

{

//Registering the delegate which will fire when we hear something

PawnSensingComp->OnHearNoise.AddDynamic(this, &AMyAICharacter::OnHearNoise);

}

void AMyAICharacter::OnHearNoise(APawn* PawnInstigator, const FVector& Location, float Volume)

{

//AMyAIController* Con = Cast<AMyAIController>(GetController());

//We don't want to hear ourselves

if (Con && PawnInstigator != this)

{

//Updates our target based on what we've heard.

//Con->SetSensedTarget(PawnInstigator);

}

You will notice that inside the OnHearNoise function I’ve commented out the lines 28 and 34. This is because we have yet to add our AI Controller class. When we’re done with our Controller, we’ll get back and uncomment that code. For now, just compile and save your code.

Creating our AI Controller

Add a new C++ class which is based on the AI Controller class and name it MyAIController. Inside the header file, before the .generated.h header file, type in the following includes:

#include "BehaviorTree/BehaviorTree.h"
#include "BehaviorTree/BehaviorTreeComponent.h"
#include "BehaviorTree/BlackboardComponent.h"

#include "BehaviorTree/BehaviorTree.h"

#include "BehaviorTree/BehaviorTreeComponent.h"

#include "BehaviorTree/BlackboardComponent.h"

Then, type in the following declarations:

protected:
	/*A Behavior tree component in order to be able to call specific functions like starting our BT*/
	UBehaviorTreeComponent* BehaviorTreeComp;

	/*A Blackboard component which will be used to initialize our Blackboard Values*/
	UBlackboardComponent* BlackboardComp;

	/*This property is used to find a certain key for our blackboard.
	We will create the blackboard later in this tutorial*/
	UPROPERTY(EditDefaultsOnly)
	FName TargetKey = "SensedPawn";

public:
	/*Default Constructor*/
	AMyAIController();

	/*Called when the AI Controller possesses a Pawn*/
	virtual void Possess(APawn* InPawn) override;

	/*Sets the new sensed target value inside our Blackboard values*/
	void SetSensedTarget(APawn* NewTarget);

protected:

/*A Behavior tree component in order to be able to call specific functions like starting our BT*/

UBehaviorTreeComponent* BehaviorTreeComp;

/*A Blackboard component which will be used to initialize our Blackboard Values*/

UBlackboardComponent* BlackboardComp;

/*This property is used to find a certain key for our blackboard.

We will create the blackboard later in this tutorial*/

UPROPERTY(EditDefaultsOnly)

FName TargetKey = "SensedPawn";

public:

/*Default Constructor*/

AMyAIController();

/*Called when the AI Controller possesses a Pawn*/

virtual void Possess(APawn* InPawn) override;

/*Sets the new sensed target value inside our Blackboard values*/

void SetSensedTarget(APawn* NewTarget);

Switch to your source file and type in the following code:

AMyAIController::AMyAIController()
{
	//Initialize our components
	BlackboardComp = CreateDefaultSubobject<UBlackboardComponent>(TEXT("BlackboardComp"));
	BehaviorTreeComp = CreateDefaultSubobject<UBehaviorTreeComponent>(TEXT("BehaviorComp"));

}

void AMyAIController::Possess(APawn* InPawn)
{
	Super::Possess(InPawn);

	//If our character is valid and has a valid Behavior Tree,
	//Initialize the values of the Blackboard and start the tree
	AMyAICharacter* Char = Cast<AMyAICharacter>(InPawn);
	if (Char && Char->BehaviorTree->BlackboardAsset)
	{
		//Initialize the blackboard values
		BlackboardComp->InitializeBlackboard(*Char->BehaviorTree->BlackboardAsset);

		//Start the tree
		BehaviorTreeComp->StartTree(*Char->BehaviorTree);
		
	}

}

void AMyAIController::SetSensedTarget(APawn* NewTarget)
{
	//Set a new target to follow
	if (BlackboardComp) BlackboardComp->SetValueAsObject(TargetKey, NewTarget);
}

AMyAIController::AMyAIController()

{

//Initialize our components

BlackboardComp = CreateDefaultSubobject<UBlackboardComponent>(TEXT("BlackboardComp"));

BehaviorTreeComp = CreateDefaultSubobject<UBehaviorTreeComponent>(TEXT("BehaviorComp"));

}

void AMyAIController::Possess(APawn* InPawn)

{

Super::Possess(InPawn);

//If our character is valid and has a valid Behavior Tree,

//Initialize the values of the Blackboard and start the tree

AMyAICharacter* Char = Cast<AMyAICharacter>(InPawn);

if (Char && Char->BehaviorTree->BlackboardAsset)

{

//Initialize the blackboard values

BlackboardComp->InitializeBlackboard(*Char->BehaviorTree->BlackboardAsset);

//Start the tree

BehaviorTreeComp->StartTree(*Char->BehaviorTree);

}

void AMyAIController::SetSensedTarget(APawn* NewTarget)

{

//Set a new target to follow

if (BlackboardComp) BlackboardComp->SetValueAsObject(TargetKey, NewTarget);

}

Moreover, inside your controller’s source file, don’t forget to add the following include:

#include “MyAICharacter.h”

When you’re done with all that, switch to your AI Character’s source file, uncomment lines 28 and 34 and add the following include:

#include “MyAIController.h”

Compile and save your code.

Setting up our Blueprints

Once you have completed all the steps above, create the following Blueprints:

A Blueprint which inherits our AICharacter class
A Blueprint which inherits our AIController class, and name it BP_MyAICon
A Behavior Tree, named MyBehaviorTree
A Blackboard named MyBlackboard

Once you’ve completed all that, go to our AI Character’s Blueprint and:

Assign the default mannequin skeletal mesh and make sure to rotate the mesh to face the blue arrow inside the capsule collider
Inside the Anim Blueprint Generated Class select the same Anim Blueprint as your main Character
Select the BP_MyAICon as the AI Controller Class
Select the MyBehaviorTree for the Behavior Tree

The above steps are summarised in the following image:

ai_character_bp — Click on the image to enlarge in a new tab

Since we don’t want a “Seeing” sense for our AI, locate it’s PawnSensingComponent and disable the See Pawns option. This is crucial.

ai_character_bp2 — Click on the image to enlarge in a new tab

Setting up our Blackboard

Our blackboard setup is simple, add a new Object key named SensedPawn.

Settin up our Behavior Tree

The logic for our AI will be the following: Once we have heard a pawn, we move to its location. When we’ve reached our target we wait. And so on… It’s not the smartest AI in the world obviously, but it will do for this tutorial. The above logic is described in the following Behavior Tree:

ai_hearing_bt2 — Click on the image to enlarge in a new tab

Moreover, click on the MoveActor node and set the acceptable radius to 100 and the Blackboard Key to SensedPawn.

Save your project and place an AI Character inside your map. The AI should now follow you like the video I’ve uploaded above.

Οrfeas

This Post Has 20 Comments

Erik says:

April 17, 2016 at 4:49 pm

I don’t know if this is a stupid question or not, but I am going to ask it anyway:
I have an AI setup going already, it has sight patrol points, deal damge etc. but it is in blueprints only, can I add this C++ to it without breaking my current system?

Thanks.

Reply
1. Οrfeas says:
  
  April 17, 2016 at 5:21 pm
  
  Depending on your actual setup you may need to do some tweaking.
  
  Since your entire AI is based on Blueprints you have already a Blueprint Controller and a Blueprint Character. In case you wish to follow my tutorial you need to create a c++ class for both the controller and the character and then reparent your Blueprints (in order to derive from the c++ classes). By doing so, you might encounter strange problems (again, depending on your Blueprint code so far) so things might go sideways.
  
  If you’re not feeling safe using C++ I think it would be a better option to implement the whole hearing system using Blueprints (it is do-able and it’s covered in an official UE4 Livestream in their youtube channel somewhere).
  
  However, you can always create a backup and revert your project if things go sideways!
  -Orfeas
  
  Reply
  1. Erik says:
    
    April 17, 2016 at 8:03 pm
    
    Thanks for your reply, I think this is a really strong tutorial, so I will go on and make a backup and try it 😀
    
    Reply
  2. Erik says:
    
    April 17, 2016 at 8:40 pm
    
    And I already ran into a problemm, I have I characterBase blueprint that manages damage and managers, but I can’t find a way to let my Enemycharacter have a C++ parent, and let that C++ parent have a characterBase parent, this characterBase is also the parent of my FPSCharacter, so I cant just give that a new parent.
    
    Reply
    1. Οrfeas says:
      
      April 17, 2016 at 10:06 pm
      
      I’m a bit confused, do you mean that you have the same parent class for both your playable character and your AI character?
      -Orfeas
      
      Reply
      1. Erik says:
        
        April 18, 2016 at 5:04 pm
        
        Yes, it basiclty manages damage and stuff like that
        
        Reply
Erik says:

April 18, 2016 at 5:04 pm

but what might work is translating that parent class to C++

Reply
1. Οrfeas says:
  
  April 18, 2016 at 8:43 pm
  
  If you manage to re-create all your logic in C++ that will definitely work
  
  -Orfeas
  
  Reply
Zcience says:

June 23, 2016 at 1:07 am

Hey,
Thanks for the tutorial!
Why are you creating a BehaviourTree Component both in the AICharacter and the AIController? Seems like you’re only using it in the AIController (and the way I’ve understood the functionality, you only need it in the Controller).

Reply
1. Οrfeas says:
  
  June 23, 2016 at 8:20 am
  
  Hello, in the AICharacter I’ve added a Behavior Tree and in the AIController I’ve added a Behavior Tree Component. The Behavior Tree located in the AICharacter contains the actual logic of the AI (notice that we assigned a Behavior Tree in the Blueprint of our character), however the Behavior Tree Component is used in order to kickstart that logic when our controller possesses a pawn.
  
  -Orfeas
  
  Reply
Alex says:

September 15, 2016 at 12:33 pm

I’m having an issue with the UPawnNoiseEmitterComponent… apparently it’s not being properly initialized in the constructor:

d:\programas (x86)\epic games\4.11\engine\source\runtime\coreuobject\public\uobject\UObject.h(87): error C2027: use of undefined type ‘UPawnNoiseEmitterComponent’

D:\Programas (x86)\Epic Games\4.11\Engine\Source\Runtime\Engine\Classes\GameFramework/Pawn.h(19): note: see declaration of ‘UPawnNoiseEmitterComponent’

C:\Users\Goncalo\Documents\Unreal Projects\CodeAI\Source\CodeAI\CodeAICharacter.cpp(43): note: see reference to function template instantiation ‘TReturnType *UObject::CreateDefaultSubobject(FName,bool)’ being compiled
2> with
2> [
2> TReturnType=UPawnNoiseEmitterComponent
2> ]

Reply
1. Οrfeas says:
  
  September 15, 2016 at 8:22 pm
  
  Hello Alex, after re-reading my published code in this tutorial, I forgot to mention that you need to add the following header file right before the .generated.h file in your character’s header file:
  #include “Perception/PawnSensingComponent.h”
  
  I think this will fix your issue, if not, please attach your code in a pastebin link
  
  -Orfeas
  
  Reply
  1. Alex says:
    
    September 15, 2016 at 8:35 pm
    
    It now works thank you :)… also could you do an AI tutorial envolving EQS? Love the stuff that you put up!
    
    Reply
    1. Οrfeas says:
      
      September 15, 2016 at 8:47 pm
      
      Thank you for your kind words and your suggestion!
      
      I’ve got an EQS tutorial on my backlog for some time now, so I think that I will take a look into that in the future.
      
      -Orfeas
      
      Reply
      1. Alex says:
        
        September 17, 2016 at 7:22 pm
        
        Hey i’m having another issue :p
        
        I’m trying to build up on your tutorials with more AI logic in which two AI characters make noises and receive each other’s noises, other than the ones that come from the player… but for some reason, they can’t hear each other’s noises!
        
        I did everything just like you did but i changed some stuff (instead of calling the ReportNoise function from the AnimBP, i call it from within the AICharacter class, and on the OnHearNoise function of the AICharacter, it also checks to see whether the same class made that noise).
        
        Any help would be greatly appreciated 😀
        
        Reply
        
        Οrfeas says:
        
        September 18, 2016 at 11:16 am
        
        Have you checked their Blackboard values in order to make sure that you have the right referenced values?
        
        Moreover, make sure that your OnHearNoise and ReportNoise function fire when they supposed to.
        
        -Orfeas
        
        Reply
        
        Alex says:
        
        September 18, 2016 at 11:54 am
        
        The problem is that one of the AICharacters is reporting the noise with the MakeNoise() function, but the other AICharacter isn’t calling the OnHearNoise() function when the noise is triggered…
        
        Reply
        
        Alex says:
        
        September 18, 2016 at 12:12 pm
        
        Whoops… “Can only sense players” was checked :p sorry about that…
        
        Reply
  2. Reece says:
    
    March 24, 2017 at 12:07 am
    
    In 4.15.1 alone did not fix my problem (I had the same problem Alex had). But once I added
    
    #include “Runtime/Engine/Classes/Components/PawnNoiseEmitterComponent.h”
    
    it worked for me then. I don’t know if Epic Games has changed the API since then but I thought I’d let you know.
    
    Awesome tutorial once again, Thank you so much for doing this! You are my favorite Unreal tutorial-maker!
    
    Reply
    1. Luiz Henrique says:
      
      January 8, 2018 at 7:54 pm
      
      That worked for me as well. I checked tom Looman’s tutorial he does the same as Orfeas. What did they do that we didn’t ? =/
      
      Reply

Implementing AI Hearing using C++

Setting up our playable Character

Creating our AI Character

Creating our AI Controller

Setting up our Blueprints

Setting up our Blackboard

Settin up our Behavior Tree

Related

Οrfeas

This Post Has 20 Comments

Leave a Reply Cancel reply

Setting up our playable Character

Creating our AI Character

Creating our AI Controller

Setting up our Blueprints

Setting up our Blackboard

Settin up our Behavior Tree

Related

Share This

Οrfeas

Related Posts

This Post Has 20 Comments

Leave a Reply Cancel reply