This is a Puppeteer script running on Apify, designed to enter values into a form and save it. The fields are an input and a textarea, respectively.
The problem is - while the values of both fields are getting successfully updated in the front end (evidenced by taking screenshots), the value of the textarea is not actually saved when the Save <button> is .clicked, even though the input value alone is saved.
Why is this happening, and how do I overcome it?
I have tried several different methods of entering the textarea text:
- Unlike the
input, I have opted against using.typefor thetextarea(even though it worked fine for small amounts of text) due to the volume of the text being entered - it was causing timeouts. - I have tried using wait times, artificially changing focus and clicking into the
textareafollowed by.typeing an arbitrary value. In no circumstance does thetextareavalue get saved. - I have read on Stackoverflow that
textareas should be edited via.innerTextrather than.value, but this seems to make no difference in my case.
FYI - the form only becomes visible and active when the user/script clicks an Edit button (before this, the page had showed the form fields as static content). After the single Edit is clicked, two Save buttons appear (.edit-video-info becomes .save-video-info At the top, and then another .save-video-info appears. But a) the script simply uses the first one found and b) the Save operation is otherwise successful, as borne out by the input value getting saved.
import { Actor } from 'apify';
import { PuppeteerCrawler } from 'crawlee'; // "puppeteer" must be included in package.json dependencies
await Actor.init();
console.log('Actor initialised.')
console.log('Welcome. Only run if input was passed...')
const input = await Actor.getInput();
if (input) {
console.log('Input was passed...')
console.log(input);
// Create an instance of the PuppeteerCrawler class - a crawler
// that automatically loads the URLs in headless Chrome / Puppeteer.
const crawler = new PuppeteerCrawler({
// Here you can set options that are passed to the launchPuppeteer() function.
launchContext: {
launchOptions: {
headless: true,
// Other Puppeteer options
defaultViewport: { width: 1800, height: 2100 }
},
},
requestHandlerTimeoutSecs: 180,
// Stop crawling after several pages
maxRequestsPerCrawl: 50,
async requestHandler({ request, page, enqueueLinks }) {
/************************************
* Function: Long-text workaround - used for body
************************************/
// workaround cf. https://github.com/puppeteer/puppeteer/issues/4192#issuecomment-475241477
async function setSelectVal(sel, val) {
page.evaluate((data) => {
return document.querySelector(data.sel).value = data.val
}, { sel, val })
}
console.log(`Processing ${request.url}`);
/************************************
* Authentication
************************************/
let isLoggedIn = false;
// Is site logged-in? true/false
const loggedCheck = async (page) => {
try {
await page.waitForSelector('input#user', { timeout: 10000 });
console.log('Logged-in: true.');
return true;
}
catch(err) {
console.log('Logged-in: false.');
return false;
}
};
isLoggedIn = await loggedCheck(page);
if (!isLoggedIn) {
console.log('Need to do login...');
const SECRET_EMAIL = 'MYEMAILADDRESS'
const SECRET_PASSWORD = 'MYPASSWORD'
await page.waitForSelector('#signinButton')
console.log('Entering email...');
await page.type('input#email', SECRET_EMAIL)
console.log('Entering password...');
await page.type('input#password', SECRET_PASSWORD)
console.log('Clicking #signinButton...');
await page.click('#signinButton')
console.log('Clicked #signinButton.');
isLoggedIn = await loggedCheck(page);
console.log('Login should now be true.');
console.log(isLoggedIn);
}
/************************************
* Kicking off...
************************************/
console.log('Do stuff when #video-details loads...');
await page.waitForSelector('#video-details', { timeout: 30000 })
// Increase browser size
await page.setViewport({
width: 1200,
height: 2400,
deviceScaleFactor: 1,
});
console.log('Existing video title...');
console.log(await page.evaluate(() => document.evaluate('normalize-space(//div[@data-test-name="video-name"]/div[2])', document, null, XPathResult.STRING_TYPE, null).stringValue));
console.log('Echoed.');
/************************************
* Edit video details...
************************************/
console.log('Clicking Edit button...');
await page.click('button[data-test-name=edit-video-info]')
console.log('Clicked Edit button.');
/************************************
* 1. Video title
************************************/
console.log('Wait for input field...');
await page.waitForSelector('label[data-test-name=video-name-input]', { timeout: 30000 })
console.log('OK.');
console.log('Changing video title...')
if (input.title) {
console.log('input.title is available')
const titlebox = await page.$('label[data-test-name=video-name-input] input');
await titlebox.click({ clickCount: 3 })
await titlebox.type(input.title);
console.log('OK')
} else {
console.log('No input.title - will not change.')
}
/************************************
* 2. Body text
************************************/
console.log('Wait for body textarea...');
await page.waitForSelector('label[data-test-name=long-description-textarea] textarea', { timeout: 30000 })
console.log('OK.');
console.log('Changing body text...')
if (input.body) {
console.log('input.body is available')
const bodytext = input.body;
console.log(bodytext);
await setSelectVal('label[data-test-name=long-description-textarea] textarea', bodytext)
console.log('OK')
} else {
console.log('No input.body - will not change.')
}
/* Screenshot */
const screen07typedinput = await page.screenshot();
await Actor.setValue('screen07typedinput.png', screen07typedinput, { contentType: 'image/png' });
/************************************
* Save video details
************************************/
// await page.focus('button[data-test-name=save-video-info]')[1];
await page.waitForTimeout(3000)
console.log('Clicking Save button...');
await page.click('button[data-test-name=save-video-info]')
console.log('Did stuff');
/* Screenshot */
const screen08aftersave = await page.screenshot();
await Actor.setValue('screen08aftersave.png', screen08aftersave, { contentType: 'image/png' });
/************************************
* Logout
************************************/
console.log('Trying logout...')
await page.goto('https://signin.example.com/logout');
console.log('Finished.');
},
// This function is called if the page processing failed more than maxRequestRetries+1 times.
failedRequestHandler({ request }) {
console.log(`Request ${request.url} failed too many times.`);
},
});
// Run the crawler and wait for it to finish.
// await crawler.run(['https://news.ycombinator.com/']);
await crawler.run(['https://site.example.com/products/videocloud/media/videos/'+input.assetid]);
console.log('Crawler finished.');
} else {
console.log('No input. Abandon.')
}
await Actor.exit();
The button markup is...
<button class="Button-blueSteel-4_1_4-3Skod Button-btn-4_1_4-QuPwe Button-primary-4_1_4-2YCmt Button-small-4_1_4-1Wurh" data-test-name="save-video-info" role="button" tabindex="0" type="button" aria-pressed="true">
<span class>Save</span>
</button>
An id attribute is absent, so I can't use getElementByID.
This works...
I had not
.clicked inside the<textarea>. Whilst I had.clicked the<input>field prior to.typeing into it, I had not done this with the<textarea>- because I wasn't using.typethere; I was having to set the `.values.In addition to that, for good measure, I
.clickBack in to the<input>field.Also, I `.types an arbitrary value.
I think it was the clicks. Without having clicked the
<textarea>, only the apparent.valuewas getting set.