Exhibit 3.4

GLADSTONE CAPITAL CORPORATION

CERTIFICATE OF CORRECTION

THIS IS TO CERTIFY THAT:

FIRST: The title of the document being corrected is Articles Supplementary (the “Articles”).

SECOND: The sole party to the Articles is Gladstone Capital Corporation, a Maryland corporation (the “Corporation”).

THIRD: The Articles were filed with the State Department of Assessments and Taxation of Maryland (the “SDAT”) on May 14, 2014.

FOURTH: The definition of “Dividend Period” in Section 10 of Exhibit A of the Articles as previously filed with the SDAT is set forth below:

“Dividend Period” means, with respect to each Series 2016 TP Share. in the case of the first Dividend Period, the period beginning on the Date of Original Issue for such Series and ending on and including June 30,2014 and for each subsequent Dividend Period. the period beginning on and including the first calendar day of the month following the month in which the previous Dividend Period ended and ending on and including the last calendar day of such month.

FIFTH: The definition of “Dividend Period” in Section 10 of Exhibit A of the Articles as corrected hereby is set forth below:

“Dividend Period” means, with respect to each Series 2021 TP Share, in the case of the first Dividend Period, the period beginning on the Date of Original Issue for such Series and ending on and including June 30, 2014 and for each subsequent Dividend Period, the period beginning on and including the first calendar day of the month following the month in which the previous Dividend Period ended and ending on and including the last calendar day of such month.

SIXTH: The undersigned acknowledges this Certificate of Correction to be the corporate act of the Corporation and as to all matters or facts required to be verified under oath, the undersigned acknowledges that to the best of his knowledge, information and belief, these matters and facts are true in all material respects and that this statement is made under the penalties for perjury.

[SIGNATURE PAGE FOLLOWS]

IN WITNESS WHEREOF, the Corporation has caused this Certificate of Correction to be signed in its name and on its behalf by its Chairman of the Board, Chief Executive Officer and President and attested to by its Internal Counsel and Secretary on May 19, 2014.