Exhibit 2.j.3

Execution Version

AMENDMENT NO. 1

TO

CUSTODIAL AGREEMENT

THIS AMENDMENT NO. 1 TO CUSTODIAL AGREEMENT (this “Amendment”) dated as of September 28, 2004, is entered into among GLADSTONE BUSINESS LOAN, LLC, as the borrower (the “Borrower”), GLADSTONE ADVISERS, INC., as the servicer (the “Servicer”), BNY MIDWEST TRUST COMPANY (the “Custodian”) and DEUTSCHE BANK AG, NEW YORK BRANCH, as successor to Canadian Imperial Bank of Commerce as Administrative Agent (in such capacity, the “Administrative Agent”). Capitalized terms used herein without definition shall have the meanings ascribed thereto in the “Custodial Agreement” or the “Credit Agreement” referred to below, as applicable.

PRELIMINARY STATEMENTS

A. Reference is made to (a) that certain Custodial Agreement dated as of May 19, 2003 among the Borrower, the Servicer, Gladstone Capital Corporation, as the originator, the Custodian and the Administrative Agent (as amended, restated, supplemented or modified from time to time, the “Custodial Agreement”) and (b) that certain Credit Agreement dated as of May 19, 2003 among the Borrower, the Servicer, the “Lenders” and “Managing Agents” from time to time party thereto and the Administrative Agent (as amended, restated, supplemented or modified from time to time, the “Credit Agreement”).

B. The parties hereto have agreed to amend certain provisions of the Custodial Agreement upon the terms and conditions set forth herein.

SECTION 1. Amendment. Subject to the satisfaction of the conditions precedent set forth in Section 2 hereof, the parties hereto hereby agree to:

(a) amend the Custodial Agreement globally to recognize (i) the resignation of Canadian Imperial Bank of Commerce (“CIBC”) as Administrative Agent under the Credit Agreement and (ii) the appointment of Deutsche Bank AG, New York Branch (“Deutsche Bank”) as successor Administrative Agent under the Credit Agreement, pursuant to that certain Resignation, Appointment and Consent dated as of even date herewith. Each reference to CIBC in the Custodial Agreement shall be amended to be a reference to Deutsche Bank, and each reference to CIBC’s address shall be deleted and the following substituted therefor:

Deutsche Bank AG, New York Branch

60 Wall Street, NYC60-1915

New York, NY 10005-2858

Attention: SPG/Gladstone Business Loan Account Officer

Telephone: (212) 250-2500

Fax: (212) 797-5150

(b) to add the following defined terms to Section 1 of the Custodial Agreement in appropriate alphabetical order therein:

“Subsequent Delivery Trust Receipt” shall mean a trust receipt issued by the Custodian pursuant to Section 3 hereof with respect to the original promissory note in respect of any Qualifying Syndicated Loan held by the Custodian for the benefit of the Administrative Agent, in substantially the form of Annex 2-C hereto.

“Subsequent Document Delivery Letter” shall have the meaning specified in Section 2(a) hereof.

(c) delete Section 2(a) of the Custodial Agreement in its entirety, and to substitute the following therefor:

(a) (A) (1) In accordance with the requirements under the Credit Agreement, the Borrower shall deliver or cause the Originator to deliver to the Custodian, as notified to the Custodian in writing with respect to each Non-Syndicated Loan, the following documents pertaining to each such Loan to be pledged to the Administrative Agent, along with a letter substantially in the form of Annex 3-A hereof (a “Primary Document Delivery Letter”) listing each document to be included in the Loan File (whether or not such document is required to be delivered pursuant to this Section 2(a)(A)) for each Loan so delivered to the Custodian, each of which Loans shall be identified in a Loan Schedule delivered in printed and electronic format therewith:

(i) the original executed promissory note issued by the Obligor pursuant to the applicable loan agreement evidencing such Loan;

(ii) an original executed counterpart of the loan agreement or comparable document relating to such Loan;

(iii) an original executed counterpart of each of the Loan Documents granting a Lien on the Collateral for such Loan; and

(iv) a copy of any financing statement filed with respect to any Collateral for such Loan; and

2

(2) Within two (2) weeks after the original delivery of any Non-Syndicated Loan, the Borrower shall deliver or cause to be delivered to the Custodian an executed original of each additional Loan Document or other document listed on Attachment B to the Primary Document Delivery Letter not previously delivered hereunder and then in the Borrower’s or the Servicer’s possession, along with a letter substantially in the form of Annex 3-B hereof (an “Ancillary Document Delivery Letter”) listing each additional document so delivered for such Non-Syndicated Loan, each of which Loans shall be identified in a Loan Schedule delivered therewith. If the Ancillary Document Delivery Letter is not delivered within two weeks, Custodian shall issue an Exception Report reporting the failure to make such delivery.

(B) (1) In accordance with the requirements under the Credit Agreement, the Borrower shall deliver or cause the Originator to deliver to the Custodian, as notified to the Custodian in writing with respect to each Qualifying Syndicated Loan, the following documents pertaining to each such Loan to be pledged to the Administrative Agent, along with a Primary Document Delivery Letter listing each document to be included in the Loan File (whether or not such document is required to be delivered pursuant to this Section 2(a)(B)) for each Loan so delivered to the Custodian, each of which Loans shall be identified in a Loan Schedule delivered in printed and electronic format therewith:

(i) a true and complete copy of the executed participation agreement or other syndication document pursuant to which the Borrower has acquired an interest in such Loan; and

(ii) a true and complete copy of the unexecuted Loan Documents in final form, as listed on Attachment B to the Primary Document Delivery Letter;

(2) Within forty-five (45) days after the initial delivery of any Qualifying Syndicated Loan, the Borrower shall deliver or cause to be delivered to the Custodian the original executed promissory note issued by the Obligor pursuant to the applicable loan agreement evidencing such Loan, or the participating financial institutions’ interest in such Loan; together with true and complete

3

copies of any additional Loan Document or other document listed on Attachment B to the Primary Document Delivery Letter then in the Borrower’s or the Servicer’s possession, along with a letter in substantially the form of Annex 3-C hereof listing each document delivered thereunder (a “Subsequent Document Delivery Letter”) listing each additional document so delivered for such Qualifying Syndicated Loan, each of which Loans shall be identified in a Loan Schedule delivered therewith. If the Subsequent Document Delivery Letter is not delivered within forty-five (45) days, Custodian shall issue an Exception Report reporting the failure to make such delivery; and

(3) Within ninety (90) days after the initial delivery of any Qualifying Syndicated Loan, the Borrower shall deliver or cause to be delivered to the Custodian a true and complete copy of each executed Loan Document or other document listed on Attachment B to the Primary Document Delivery Letter not previously delivered hereunder and then in the Borrower’s or the Servicer’s possession, along with an Ancillary Document Delivery Letter listing each additional document so delivered for such Qualifying Syndicated Loan, each of which Loans shall be identified in a Loan Schedule delivered therewith. If the Ancillary Document Delivery Letter is not delivered within ninety (90) days, Custodian shall issue an Exception Report reporting the failure to make such delivery.

(d) amend Section 3 of the Custodial Agreement to add the following clause (c) in appropriate alphabetical order therein and to reletter the subsequent clauses appropriately:

(c) Following receipt of a Subsequent Document Delivery Letter, but in no event later than 1:00 p.m. (New York time) on the date which is three (3) Business Days after receipt of the original promissory note for any Loan File for a Qualifying Syndicated Loan, the Custodian will deliver, via facsimile, to the Administrative Agent, with a copy to the Servicer, a Subsequent Delivery Trust Receipt to the effect that, as to such Qualified Syndicated Loan listed on the related Loan Schedule, based on the Custodian’s examination of the Loan File for such Qualifying Syndicated Loans, except for Exceptions noted in an Exception Report, (i) the original promissory note required to be delivered in

4

respect of such Qualifying Syndicated Loan pursuant to Section 2(a) of this Custodial Agreement has been delivered and is in the possession of the Custodian as part of the Loan File for such Qualifying Syndicated Loan and (ii) such promissory note has been reviewed by the Custodian and appears on its face to be regular, to relate to the applicable Qualifying Syndicated Loans and to satisfy the applicable requirements set forth in Section 2(a) of this Custodial Agreement. In connection with a Subsequent Delivery Trust Receipt, the Custodian shall make no representations as to and shall not be responsible to verify (A) the validity, legality, enforceability, due authorization, recordability, sufficiency, or genuineness of the documents contained in each Loan File or (B) the collectability, insurability, effectiveness or suitability of any such Qualifying Syndicated Loan.

(e) To amend Annex 2 to add the form attached hereto as Exhibit A as Annex 2-C in appropriate alphabetical order therein.

(f) To delete Annex 6 in its entirety and substitute the form attached hereto as Exhibit B therefor.

SECTION 2. Conditions Precedent. This Amendment shall become effective on the first Business Day (the “Effective Date”) on which the Administrative Agent and the Custodian have received a fully executed counterpart of this Amendment.

SECTION 3. Reference to and Effect on the Transaction Documents.

(a) Upon the effectiveness of this Amendment, (i) each reference in the Custodial Agreement to “this Custodial Agreement”, “this Agreement”, “hereunder”, “hereof”, “herein” or words of like import shall mean and be a reference to the Custodial Agreement as amended or otherwise modified hereby, and (ii) each reference to the Custodial Agreement in any other Transaction Document or any other document, instrument or agreement executed and/or delivered in connection therewith, shall mean and be a reference to the Custodial Agreement as amended or otherwise modified hereby.

(b) Except as specifically amended, terminated or otherwise modified above, the terms and conditions of the Custodial Agreement and any other documents, instruments and agreements executed and/or delivered in connection therewith, shall remain in full force and effect and are hereby ratified and confirmed.

(c) The execution, delivery and effectiveness of this Amendment shall not operate as a waiver of any right, power or remedy of the Administrative Agent or the Custodian under the Custodial Agreement or any other Transaction Document or any other document, instrument or agreement executed in connection therewith, nor constitute a waiver of any provision contained therein, in each case except as specifically set forth herein.

5

SECTION 4. Execution in Counterparts. This Amendment may be executed in any number of counterparts and by different parties hereto in separate counterparts, each of which when so executed and delivered shall be deemed to be an original and all of which taken together shall constitute but one and the same instrument. Delivery of an executed counterpart of a signature page to this Amendment by electronic delivery shall be effective as delivery of a manually executed counterpart of this Amendment.

SECTION 5. Governing Law. This Amendment shall be governed by and construed in accordance with the laws of the State of New York.

SECTION 6. Headings. Section headings in this Amendment are included herein for convenience of reference only and shall not constitute a part of this Amendment for any other purpose.

[Remainder of Page Deliberately Left Blank]

6

IN WITNESS WHEREOF, the parties hereto have caused this Amendment to be duly executed by their respective officers as of the date first above written.

Signature Page to Amendment No. 1

Signature Page to Amendment No. 1