Stačí přidat další HTTP direktivy:
<IfModule mod_rewrite.c>
RewriteEngine on
RewriteCond %{HTTPS} !=on
RewriteRule ^(.*)$ https://%{HTTP_HOST}/$1 [R,QSA]
</IfModule>
více zde:
https://hanssens.com/trials-and-tribulations/redirect-http-to-https-for-a-domain-in-plesk-12-5/