I've seen a Mortar addon somewhere ... have you looked at that?
Looked at the mortar, added a smoke shell, didnt work, edited it more, didnt work ......
Looked at other addons, no one seems to have beat the system.
Closest I can find is the SD_BTR80 addon
But they script the action of launching smoke from the main gun (swap out the weapon, fire "smoke weapon", swap back the normal weapon), then used a FIRED eventhandler to catch the "shell" and script launch some smoke grenades instead.
You could do something similar by having a normal (customize, little to no explosion effects) shell fire, use the FIRED eventhandler to watch the shell. When the shell terminated / expires, replace it with a smoking custom grenade or similar .......
Sorry, after 3 hrs rooting around I can't see a config only solution.
........ OFP used to at least have smoke and flare launchers