In this article, response surface methodology (RSM) has been applied to simultaneously optimize the tensile and impact strength of polypropylene (PP)/nitrile butadiene rubber (NBR)/halloysite nanotubes (HNTs)/maleic anhydride (MA)‐grafted‐PP nanocomposites. Three levels of material parameters, including NBR (10, 20, and 30 wt%), HNTs (1, 3, and 5 wt%), and PP‐g‐MA (3, 9, and 15 wt%) as compatibilizers were used to design the experiments according to Box‐Behnken design. In order to investigate the morphology of nanocomposite samples, a scanning electron microscope was used. The model predicted that both the tensile strength and impact strength of nanocomposite were peaked between middle and high levels of HNTs content. There was also a peak for NBR loading in the main effect plot of impact strength. Two‐way interactions of all parameters were significantly affecting both responses. Based on RSM models using the desirability function, the optimum values of input parameters leading to optimized tensile strength and impact strength were predicted to be 17.87, 4.35, and 15 wt% for NBR, HNT, and PP‐g‐MA, respectively. Confirmation experiments were in good agreement with the predicted values.