Code With Bisky

Mastering Real-Time Collaboration: Flutter WebRTC Screen Sharing Guide Tutorial [27]

Key Topics covered:

  • Flutter WebRTC
  • Initiate Video Call
  • Listening to An Incoming Video Call

Description:

In this video, we'll walk you through the process of implementing screen sharing functionality in your Flutter applications

Code Snippet(pubspecyaml.dart):


dependencies:
    ........
  flutter_background: ^1.2.0

 

Code Snippet(VideoCallState.dart):

Add sharingScreen state


@freezed
class VideoCallState with _$VideoCallState {
  factory VideoCallState({
                .......
    @Default(false) bool sharingScreen,
  }) = _VideoCallState;
}

 

Code Snippet(RoomAppwrite.dart):

Add presentingUserId field and also add it to the rooms collection on Appwrite dashboard


  String? presentingUserId;
  RoomAppwrite({....,this.presentingUserId});

 

Code Snippet(RoomRepositoryProvider.dart):

  • lib/core/providers
  • RoomRepositoryProvider.dart

Code Snippet(RoomRepositoryProvider.dart):

Let's add method to update user who is presenting the screen


 Future<RoomAppwrite?> updatePresenterUserRoom(RoomAppwrite room,String? presenterUserId) async {
    try {
        Document document = await _db.updateDocument(
            databaseId: Strings.databaseId,
            collectionId: Strings.collectionRoomId,
            documentId: room.roomId!,
            data: {
              'presentingUserId': presenterUserId
            });
        return RoomAppwrite.fromJson(document.data);


      return RoomAppwrite.fromJson(document.data);
    } catch (e) {
      print('ERROR updatePresenterUserRoom $e');
    }
    return null;
  }
 

Run the following command flutter packages pub run build_runner build

We need to mdify below classes

  • lib/pages/dashboard/video_calls/one_to_one
  • VideoCallViewModel.dart
  • VideoCallVMScreen.dart

Code Snippet(VideoCallViewModel.dart):


import 'package:chat_with_bisky/core/providers/UserRepositoryProvider.dart';

class VideoCallViewModel extends _$VideoCallViewModel {
........

/ ignore: avoid_public_notifier_properties
  UserDataRepositoryProvider get userRepository =>
// ignore: avoid_manual_providers_as_generated_provider_dependency
  ref.read(userRepositoryProvider);

// ignore: avoid_public_notifier_properties
  ValueChanged<String>? onUserPresenting;



// on room listeners methods, cancel screen sharing is another user is still sharing
cancelScreenSharingListener(roomState);



  void cancelScreenSharingListener(RoomAppwrite roomState) {
    if(state.sharingScreen && roomState.presentingUserId != state.myUserId){
      cancelShareStream();
    }
  }


........
}

 

Share Screen and Cancel Screen Methods(VideoCallViewModel.dart):



// add below methods to handle screen sharing
Future<void> createShareStream(bool force) async {

    RoomAppwrite? room = await _roomRepository.getRoom(state.roomId??"");
    if(room == null){
      return;
    }
    if(room.presentingUserId != null && force == false){
      UserAppwrite? user = await userRepository.getUser(room.presentingUserId!);
      if(user!=null){
        onUserPresenting!("${user.name ?? room.presentingUserId!} is presenting the screen");
      }else{
        onUserPresenting!("${room.presentingUserId!} is presenting the screen");
      }
      return;
    }

    final Map<String, dynamic> mediaConstraints = {
      'audio': true,
      'video': true
    };

    if (WebRTC.platformIsIOS) {
      mediaConstraints['video'] = {'deviceId': 'broadcast'};
    }
    MediaStream stream =    await navigator.mediaDevices.getDisplayMedia(mediaConstraints);
    MediaStreamTrack? newTrack = stream.getTracks().where((element) => element.kind == 'video').firstOrNull;
    if (newTrack!= null) {
      List<RTCRtpSender>? senders = await peerConnection?.getSenders();
      if (senders != null) {
        senders.forEach((s) async {
          if (s.track != null && s.track?.kind == 'video') {
            await s.replaceTrack(newTrack);
          }
        });
        if (onLocalStream != null) {
          onLocalStream!(stream);
          setSharing(true);
          _roomRepository.updatePresenterUserRoom(room, state.myUserId);
        }
      }
    }
  }

  Future<void> cancelShareStream() async {
    MediaStream stream =    await createStream();
    MediaStreamTrack? newTrack = stream.getTracks().where((element) => element.kind == 'video').firstOrNull;

    if (newTrack != null) {
      List<RTCRtpSender>? senders = await peerConnection?.getSenders();
      if (senders != null) {
        senders.forEach((s) async {
          if (s.track != null && s.track?.kind == 'video') {
            await s.replaceTrack(newTrack);
          }
        });
      }
      setSharing(false);
      RoomAppwrite? room = await _roomRepository.getRoom(state.roomId??"");
      if(room != null){
        _roomRepository.updatePresenterUserRoom(room, null);
      }
    }
  }

   void cancelScreenSharingListener(RoomAppwrite roomState) {
    if(state.sharingScreen && roomState.presentingUserId != state.myUserId){
      cancelShareStream();
    }
  }

 

Close remote streams(VideoCallViewModel.dart):



 closeStreams() {
  .....
   _remoteStreams?.forEach((element) {
      element.dispose();
    });
    peerConnection?.dispose();
    hangupSub?.cancel();
}

 

Assigning peerConnection(VideoCallViewModel.dart):




  joinRoom(String sessionDescription, String sessionType) async {
                .......
      this.peerConnection = peerConnection;
                ........
  }
 

Code Snippet(VideoCallVMScreen.dart):



// delete deactivate method
 ---  deactivate(){}

// dispose _localRenderer and in dispose() method before calling super.dispose();
void dispose() {
.......
if (_notifier != null) _notifier?.closeStreams();
    _localRenderer.dispose();
    _remoteRenderer.dispose();
    ref.invalidate(videoCallViewModelProvider);
}


 // add a share screen button and add heroTags to other FloatingActionButtons
   _state!.isOnCall
            ? FloatingActionButton(
            heroTag: "btn2",
            backgroundColor: Colors.blue,
            onPressed: () {

              if(_state!.sharingScreen == true){
                _notifier?.cancelShareStream();
              }else{
                _notifier?.createShareStream(false);
              }

            },
            child: Icon(_state!.sharingScreen ? Icons.cancel_presentation : Icons.present_to_all),
          ): const SizedBox(),

// add user presenting lister in callsEventListeners() method
 _notifier?.onUserPresenting = ((presentingMessage) {
      _modalAnotherUserSharingBottomSheet(presentingMessage);
    });

// add a bottom dialog to force presenting if other use is presenting
void _modalAnotherUserSharingBottomSheet(String message) {
    showModalBottomSheet(
      context: context,
      builder: (builder) {
        return Container(
          height: 250.0,
          decoration: const BoxDecoration(
            color: Colors.white,
            borderRadius: BorderRadius.only(
                topLeft: Radius.circular(10), topRight: Radius.circular(10)),
          ),
          child: Container(
            width: double.infinity,
            color: Colors.white,
            child: Column(
              children: [
                const SizedBox(
                  height: 10,
                ),
                Text(message),
                const SizedBox(
                  height: 10,
                ),
                InkWell(
                  onTap: () {
                    Navigator.pop(context);
                    _notifier?.createShareStream(true);
                  },
                  child: Container(
                    alignment: Alignment.center,
                    width: double.infinity,
                    height: 38,
                    child: const Row(
                      crossAxisAlignment: CrossAxisAlignment.center,
                      mainAxisAlignment: MainAxisAlignment.center,
                      children: [
                        SizedBox(
                          height: 10,
                        ),
                        Icon(
                          Icons.video_camera_front,
                          color: Colors.grey,
                        ),
                        Text("Continue Sharing"),
                        Spacer()
                      ],
                    ),
                  ),
                ),
              ],
            ),
          ),
        );
      },
    );
  }
 

Code Snippet(main.dart):

We need to configure flutter_background so that will be able to handle screen sharing on Android Devices


import 'package:flutter_webrtc/flutter_webrtc.dart';
import 'package:flutter_background/flutter_background.dart';


// let's add below code in the method initializeService() at first line

  if (WebRTC.platformIsAndroid) {
    final androidConfig = const FlutterBackgroundAndroidConfig(
    notificationTitle: "CodeWithBisky",
    notificationText: "Background notification for keeping the CodeWithBisky running in the background",
    notificationImportance: AndroidNotificationImportance.Default,
    notificationIcon: AndroidResource(name: 'background_icon', defType: 'drawable'), // Default is ic_launcher from folder mipmap
  );

    await FlutterBackground.initialize(androidConfig: androidConfig).then(
        (value) async {
      // value is false
      if (value) {
         bool enabled = await FlutterBackground.enableBackgroundExecution();
      }
      return value;
    }, onError: (e) {
      print('error FlutterBackground.initialize $e');
    });
  }
 

Code Snippet(AndroidManifest.xml):

We need to add below permissions and a service to handle screen sharing


                <uses-permission android:name="android.permission.WAKE_LOCK" />
                <uses-permission android:name="android.permission.FOREGROUND_SERVICE"/>
                <uses-permission android:name="android.permission.REQUEST_IGNORE_BATTERY_OPTIMIZATIONS" />

                // add this in your project application. The service is from flutter_background dependency we added
                 <service android:name="de.julianassmann.flutter_background.IsolateHolderService"
                          android:foregroundServiceType="mediaProjection"
                          android:enabled="true"
                          android:exported="false"/>

 

With these configurations, android devices would be able to share screens. The next section is to add iOS configurations

iOS Screen Sharing:

We need to configure iOS devices screen sharing. It is complicated, but we will walk you through the process

Create a BroadCast Extension target:

Open your application with xcode. We need to create a group first so that the application Runner and Broadcast Extension would be in same group.

Select Runner >Signing & Capabilities > Select Target Runner > Click + Capability. Search for App Groups and enter you group like this one group.com.chat.with.bisky.chatWithBisky

Create iOS Group

Create Broadcast Extension:

Click File> New > Target > Choose Broadcast Upload Extension

Broadcast Upload Extension Broadcast Extension

Include UI Extension should be false. Don't tick it. Add the following files into the folder Broadcast Extension.Atomic.swift, Broadcast Extension.entitlements, DarwinNotificationCenter.swift,SampleUploader.swift,SocketConnection.swift and replace SampleHandler.swift contents

Get files from the following link

Broadcast Extension Files

Update appGroupIdentifier in SampleHandler.swift that you created. Make sure this extension is in the same group with your Runner. The group must match. Do the same steps that you did to create a group for a Runner on this Broadcast Extension Target

Add below in your Application Info.plst:


                <key>RTCAppGroupIdentifier</key>
                <string>group.com.chat.with.bisky.chatWithBisky</string>
                <key>RTCScreenSharingExtension</key>
                <string>com.chat.with.bisky.chatWithBisky.Broadcast-Extension</string>
                <key>UIBackgroundModes</key>
                <array>
                        <string>remote-notification</string>
                        <string>voip</string>
                        <string>audio</string>
                </array>
 

Add below in Extension Info.plst:



                <key>com.apple.security.application-groups</key>
                <array>
                    <string>group.com.chat.with.bisky.chatWithBisky</string>
                </array>
 

Conclusion:

In a nutshell, incorporating screen sharing with Flutter WebRTC adds a whole new dimension to your app's capabilities. From interactive presentations to seamless remote work, the potential is immense. By following our tutorial, you've gained the tools to make it happen. Now, go ahead and elevate your app with the magic of real-time screen sharing!

Thank you for joining us on this exciting coding adventure! If you found this blog valuable, don't forget to share, and subscribe to our YouTube channel for more engaging Flutter content. Keep coding, keep innovating, and keep pushing the boundaries of what you can create.