To record the audio, Android provide a class "MediaRecorder" , you just need to create its instance.

Step 1:-
In your xml, take two buttons to start and stop recording.

Step 2:-
Declare the needed variables

private MediaRecorder audioRecorder;
private boolean isRecording = false,
Button startRecordButton,stopButton;
private String outFile=Environment.getExternalStorageDirectory().getAbsolutePath()+ "/recording.mp3";

Step 3:- Include methods on your main Activity

Here you need to specify the following:-

setAudioSource(int):- defines both a default physical source of audio signal, and a recording configuration.
setAudioEncoder(int):- If this method is not called, the output file will not contain an audio track.
setOutputFormat(int):-Sets the format of the output file produced during recording.

(a) Call the below function , on the click event of start button

private void startRecorder()
{
if (!isRecording) {
    audioRecorder = new MediaRecorder();
    audioRecorder.setAudioSource(MediaRecorder.AudioSource.MIC);
    audioRecorder.setOutputFormat(MediaRecorder.OutputFormat.THREE_GPP); 
    audioRecorder.setAudioEncoder(MediaRecorder.OutputFormat.AMR_NB);
    audioRecorder.setOutputFile(outFile);
    isRecording = true;
    try {
        audioRecorder.prepare();
        audioRecorder.start();
    } catch (IllegalStateException e) {
        e.printStackTrace();
    } catch (IOException e) {
        MyUtilities.showLog("Exception AudioRecorder");
        e.printStackTrace();
    }
    Toast.makeText(getApplicationContext(), "Recording started", Toast.LENGTH_LONG).show();
} else {
    stopAudioRecorder();
}}

(b) Call the below function , on the click event of stop button

After stopping the audio recorder, one must release the resource associated with recorder.

private void stopAudioRecorder() {
    isRecording = false;
    audioRecorder.stop();
    audioRecorder.release();
    audioRecorder = null;
    Toast.makeText(getApplicationContext(), "Audio recorded successfully", Toast.LENGTH_LONG).show();
}

Step:-4 you need to add runtime permission for accessing external storage and  mic
a) Create a function to make request

private void requestPermission() {
    if (ContextCompat.checkSelfPermission(CustomMainScreen.this, Manifest.permission.WRITE_EXTERNAL_STORAGE) != PackageManager.PERMISSION_GRANTED) {
        ActivityCompat.requestPermissions(CustomMainScreen.this, new String[]{Manifest.permission.WRITE_EXTERNAL_STORAGE},
                Constants.MY_PERMISSIONS_REQUEST);
    } else if (ContextCompat.checkSelfPermission(CustomMainScreen.this, Manifest.permission.READ_EXTERNAL_STORAGE) != PackageManager.PERMISSION_GRANTED) {
        ActivityCompat.requestPermissions(CustomMainScreen.this, new String[]{Manifest.permission.READ_EXTERNAL_STORAGE},
                Constants.MY_PERMISSIONS_REQUEST);
    } else if (ContextCompat.checkSelfPermission(CustomMainScreen.this, Manifest.permission.RECORD_AUDIO) != PackageManager.PERMISSION_GRANTED) {
        ActivityCompat.requestPermissions(CustomMainScreen.this, new String[]{Manifest.permission.RECORD_AUDIO},
                Constants.MY_PERMISSIONS_REQUEST);
    } else {
        ActivityCompat.requestPermissions(CustomMainScreen.this, new String[]{Manifest.permission.WRITE_EXTERNAL_STORAGE, Manifest.permission.READ_EXTERNAL_STORAGE, Manifest.permission.RECORD_AUDIO},
                Constants.MY_PERMISSIONS_REQUEST);
    }
}

b) override the onRequestPermissionsResult(int requestCode, String permissions[], int[] grantResults)

public void onRequestPermissionsResult(int requestCode, String permissions[], int[] grantResults) {
        if (requestCode == Constants.MY_PERMISSIONS) {
            for (int i = 0; i < permissions.length; i++) {
                if (grantResults.length > 0 && grantResults[0] == PackageManager.PERMISSION_GRANTED) {
               /*once permission granted ...code according your requirement*/
                } else {
                    Toast.makeText(CustomMainScreen.this, permissions[i] + " not granted", Toast.LENGTH_SHORT).show();
                }
            }
        }
        return;
    }

Note :- As you need to add the runtime permission to access recording in "Marshmallow".

To get the runtime permission , please follow the link to get more clear on the permissions-access:-
http://findnerd.com/account/#url=/list/view/How-to-request-permissions-in-Android-marshmallow-at-run-time-/14378/