The contribution of the current study is to investigate the mixed convection in an inclined nanofluid filled cavity saturated with a partially layered non-Darcy porous medium. Moreover, due to the advantage of the particle-based methods, we presented the improved version of an incompressible smoothed particle hydrodynamics (ISPH) method. The current ISPH method was improved in boundary conditions treatment using renormalization kernel function. In the current investigation, we assumed that the inclined cavity is filled with a Cu-water nanofluid. The upper half of the cavity is saturated with a non-Darcy porous medium. Here, one domain approach is used for coupling the nanofluid and the porous medium layer. The cooled top wall of the cavity is carrying a tangential unit velocity and the bottom wall is heated. The other two wall sides are adiabatic at zero velocity. Here, we investigated the effects of the Richardson parameter $Ri0.0001\u2013100$, Darcy parameter Da $10\u22125\u201310\u22122$, an inclination angle $\alpha 0\u201390deg$ and a various solid volume fraction $\varphi 0\u20130.05$ on the heat transfer of a Cu-water nanofluid. The obtained results showed that the average Nusselt number decreases as the Richardson number increases. An addition of 1–5% Cu nanoparticles slightly increased the overall heat transfer rate.