I've been using the Aggregator plugin for a while now for a number of functions, mainly for deduping. Right now I am trying to use it in a DSS task over approx 838K rows but get this error:
TE_7041 Aggregate Error: Failed to allocate 249344 bytes
I've tried this pointed at two different server targets under the assumption it was related to drive space and temp files. Any suggestions on how to resolve this? Any Informatica Cloud settings that can be adjusted to accommodate?
There may be couple of reasons the secure agent cannot address more memory. Environmental constrains: OS, Machine, and memory available. As you move large data set.
Have you tested this successfully in PowerCenter first?
Another best practice is to use the Sorter Transformation in front of the aggregator, then mark the Aggregator transform as to have sorted inputs under the Properties Tab of the Aggregator in PowerCenter. You want to sort the column in the sorter transformation based on columns you are grouping by in the Aggregator. Take out any extra aggregate functions that are not needed.
When running this in ICS, gradually bump up the memory setting. See picture below.
The setting is found under the Agent Sub-Tab, Edit Agent, Under Configuration Details section choose Type: Tomcat JRE.
Hope this helps keep us posted.
Thanks for this very helpful response!
I don't have access to PowerCenter so this has not been tested there. Therefore also not able to run the Sorter transformation (or is it available as a IC plugin?)
Hi Elias...have given this a try, no luck so far. I've upped it to 2gb so far. Do you have anythoughts or documentation on any of the other configuration options I see there, like JRE_OPTS?
Hi Elias...as a matter of fact I did some pretty extensive testing to find the limit and that came out to about 400k. So for the project in question I broke 1 task into 3 tasks with that in mind i.e. < 400k but with enough of a buffer for future record growth. Then I'll use a post processing script to combine the 3 targets into 1 target. Definitely a work around but gets the job done.