We will include your request for an appropriate option, e.g. "-yesnocancel", in the GuiXT wish list.
To work around it now... I suggest you use the "/O" status overview screen. The InputScript is as follows:

Enter "/O"

Screen rsm04000_alv.2000

  Title "Exit Project"

  // del existing elements
  del (0,0) (20,200)
  del P[/0]
  del P[/5]
  del P[/14]

  Text (1,10) "The data entered will be lost."
  Text (2,10) "Do you want to save the data?"

  Pushbutton (4,10) "Yes" "/12" process="project_continue.txt" size=(1,8)
  using CODE = "YES" 

  Pushbutton (4,22) "NO"  "/12" process=project_continue.txt" size=(1,8)
  using CODE = "NO" 

  Pushbutton (4,36) "CANCEL"  "/12" process="project_continue.txt" size=(1,8)
  using CODE = "CANCEL" 

  On "/12" fcode="/12" process="project_continue.txt"
  using CODE = "CANCEL" 

  On "/0" "/12" process="project_continue.txt"
  using CODE = "CANCEL" 

and a further InputScript "project_continue.txt" of the following structure

Parameter CODE

if U[CODE=CANCEL]
  Leave
endif

if U[CODE=NO]
  goto start_action
endif

// save
Screen ...
  Message "Save"  -yesno   // test output
  Enter

label  start_action
Screen ...
  Message "Action"   -yesno  // test output
  Enter