Answer :

Solve the homogeneous equation

[tex]y'' + 3y' + 2y = 0[/tex]

Its characteristic equation is

[tex]r^2 + 3r + 2 = (r + 1) (r + 2) = 0[/tex]

with roots at [tex]r=-1[/tex] and [tex]r=-2[/tex], hence the characteristic solution is

[tex]y_c = C_1 e^{-x} + C_2 e^{-2x}[/tex]

For the nonhomogeneous equation, I'll use variation of parameters. We're looking for a solution of the form

[tex]y = u_1 y_1 + u_2 y_2[/tex]

to the equation

[tex]y'' + a(x) y'' + b(x) y = f(x)[/tex]

such that

[tex]\displaystyle u_1 = - \int \frac{y_2f(x)}{W(y_1,y_2)} \, dx[/tex]

[tex]\displaystyle u_2 = \int \frac{y_1 f(x)}{W(y_1,y_2)} \, dx[/tex]

The Wronskian [tex]W(y_1,y_2)[/tex] of the two fundamental solutions [tex]y_1=e^{-x}[/tex] and [tex]y_2=e^{-2x}[/tex] is

[tex]W(y_1,y_2) = \begin{vmatrix} y_1 & y_2 \\ {y_1}' & {y_2}' \end{vmatrix} = -e^{-3x}[/tex]

Then we have

[tex]\displaystyle u_1 = - \int \frac{e^{-2x} \cdot 4e^x \cos(3x)}{-e^{-3x}} \, dx = 4 \int e^{2x} \cos(3x) \, dx[/tex]

[tex]\displaystyle u_2 = \int \frac{e^{-x} \cdot 4e^x \cos(3x)}{-e^{-3x}} \, dx = -4 \int e^{3x} \cos(3x) \, dx[/tex]

Recall Euler's identity,

[tex]e^{(a+bi)t} = e^{at} (\cos(bt) + i \sin(bt))[/tex]

Then we have the general antiderivative

[tex]\displaystyle \int e^{(a+bi)t} \, dt = \frac1{a+bi} e^{(a+bi)t} + C = \frac{a-bi}{a^2+b^2} e^{(a+bi)t} + C[/tex]

Taking the real parts of both sides, we have

[tex]\displaystyle \mathrm{Re}\left\{\int e^{(a+bi)t} \, dt \right\} = \mathrm{Re}\left\{\frac{a-bi}{a^2+b^2} e^{(a+bi)t} + C\right\} \\\\ \int\,\mathrm{Re}\left\{e^{(a+bi)t}\right\} \, dt = \frac{e^{at}}{a^2+b^2} \mathrm{Re}\left\{(a-bi)(\cos(bt) + i \sin(bt))\right\} + C \\\\ \int e^{at} \cos(bt) \, dt = \frac{e^{at}}{a^2+b^2} (a\cos(bt)+b\sin(bt)) + C[/tex]

so that

[tex]\displaystyle u_1 = 4 \int e^{2x} \cos(3x) \, dx = \frac{4e^{2x}}{13} (2\cos(3x) + 3 \sin(3x))[/tex]

and

[tex]\displaystyle u_2 = -4 \int e^{3x} \cos(3x) \, dx = -\frac{2e^{3x}}3 (\cos(3x) + \sin(3x))[/tex]

We've found

[tex]y = u_1 y_1 + u_2 y_2[/tex]

[tex]\displaystyle y = \frac{4e^x}{13} (2\cos(3x) + 3 \sin(3x)) - \frac{2e^x}3 (\cos(3x) + \sin(3x))[/tex]

[tex]\displaystyle y = \frac2{39} e^x (5\sin(3x) - \cos(3x))[/tex]

Then the general solution to the differential equation is

[tex]\boxed{y(x) = C_1 e^{-x} + C_2 e^{-2x} + \frac2{39} e^x (5\sin(3x) - \cos(3x))}[/tex]