When AOI Flags a Defect: Match the Location First, Then Separate Real from False, Then Trace the Cause Upstream

An AOI defect call is just the starting point. First confirm which location on the board it refers to, then confirm whether it is a real defect or a false call, and after that trace back to the station that caused it. If the first two steps are not done well, what you investigate afterward may be a false signal.

A note on sources and limits first. This article draws on older internal records we have referred to before: a 2012 analysis report on solder non-wetting on a fuse (including records of AOI overcalling), a 2010 AOI trial evaluation with four sets of repeat-test data, an internal explanation of how AOI makes its judgments, a 2014 weekly report, and a 2009 list of improvement requests submitted to a placement machine maker. Companies, downstream customers, names, part numbers and machine models are not given. The records are old and the machines and products all differ; what we borrow is how to handle an AOI defect call, not any figure as a standard value. For public sources, we read the first few pages of two IPC standards and one trade magazine article; what we did and did not read is listed at the end.

1. Case: AOI flagged it, but the real defect still shipped

Q: What happened in this case?

A: AOI could detect the defect; the problem came after the call, when a person transferred the location to the board and got it wrong. The 2012 report began with a downstream report of electrical failures. X-ray images and cross-sections showed that the clearly defective samples all had the same component (a fuse) with solder non-wetting. After reworking that fuse, the retest passed, confirming it as the cause of the electrical failure.

The report went through the 4Ms (material, machine, man, method) one by one. Under material, the component maker said non-wetting on this fuse was about 10,000 dppm, while the other capacitors were all below 300 dppm, and suspected heavier oxidation on the solderable surface. Under machine, it says: tested with samples, AOI could detect the non-wetting with a 100% detection rate, and the AOI program was set toward overcalling, with no missed calls. In other words, the machine did not miss it.

Method was the key: this product had 336 units per panel, while other card types in the same plant had at most 48 or 30 per panel. With AOI set to overcall, there were even more locations to mark on each panel. The report’s inference (the report says “highly likely,” not confirmed) was that the operator put the red marker on the wrong unit (which was actually good) and missed marking the one that was really bad. The repair technician checked the marked unit, found nothing, and the panel moved on to the next station.

Figure: AOI call locations depend on a person transferring them to the board — two scenarios, marking correctly and marking one wrong (illustration; the numbers are the report’s reasoning, not measurements).

The report’s countermeasures: build a fixture so operators can more easily confirm the location of a defective unit; add a numbered ruler on the inline AOI screen to match the numbered ruler on the board; add an AOI-2 step and a re-inspection to the process and update the routing card; and train operators on the new method.

2. Be clear about “rates” first: false call rates can be calculated several ways

Q: What do “false call,” “overcall” and “escape” mean?

A: When AOI reports a problem that is not actually there, it is a false call (also called overcall or overkill). When there is a real problem that AOI does not report, it is an escape (also called underkill). This is how this article uses the terms; usage varies between companies.

Q: Why must a false call rate always state its denominator?

A: With different denominators, the same machine can look very good or very bad. We have two old records, both called “false calls,” calculated in completely different ways:

Figure: both are called “false calls,” but different denominators give very different numbers (illustration; figures from old internal records; machines, products and years all differ, so they cannot be compared).

These two figures cannot be compared with each other: the machines, products and years differ, and even how a “point” is counted differs. IPC standards warn about this. IPC-9261A, In-Process DPMO and Estimated Yield for PWAs (October 2006), defines DPMO as the number of defects divided by the number of defect opportunities, multiplied by one million, and explains that it is not exactly the same as PPM because “opportunities” are counted in a particular way in electronics assembly (for example, a component counts as one opportunity regardless of its number of leads). It also notes that the standard assumes 100% inspection efficiency, and that care is needed when comparing manual inspection with automated optical inspection. IPC-7912A, End-Item DPMO for Printed Circuit Board Assemblies (January 2004), cautions that comparing indices between companies needs care because assembly complexity and data volume differ; it deals with defects found on finished items for benchmarking, not for tracking defect types or driving improvement. So we use them just as the basis for “state the denominator when you report a rate,” not as thresholds for AOI false call rates.

Also, in the 2010 trial evaluation table, the point count column says 2,242 components, so the denominator appears to be counted by component; we have not verified whether its method follows the IPC definition.

3. Step one: match the location — the one AOI means must be the one a person finds

Q: How do you confirm that the location a person picks up is the same one AOI flagged?

A: Treat “the location on the AOI screen” and “the location on the board” as two sets of data, and have a way to match them entry by entry. This is exactly where the fuse case broke down: 336 units per panel, locations transferred by hand from screen to board, so one wrong mark let a real defect through — and the repair side even concluded “checked, nothing wrong.” The report’s fix was to add a fixture and numbered rulers so the locations shared common coordinates.

Another 2010 AOI trial comparison table lists “NG component marking system” among its specification items, and for that machine it says “none,” meaning the machine does not mark defect locations on the board itself. That was a different machine and cannot be assumed for the one in the fuse case, but this item is worth asking about once during selection and acceptance.

Our summary (not taken from the records, and not a standard):

• When a panel has many units, the AOI screen and the board should use the same numbering or coordinates.
• Repair feedback should not just say “no problem”; it should say “which location I looked at and what I saw.”
• When repair reports “nothing found,” first confirm the marked location was right, then decide whether it was a false call.
• Sample a few flagged locations and check them under a microscope against the AOI location, so that “marked location = AOI location” becomes something you can check.

4. Step two: separate real from false — keep records of re-inspection and guard against fatigue

Q: What goes wrong in re-inspection when there are many false calls?

A: Real defects get buried among false calls. An electronics manufacturing magazine article (SMT007: “Realizing the Expectations of AOI”; the author’s affiliation is a company whose business the page does not describe; the date shown on the page is invalid, and from its mention of lead-free processes and the start of 0201 volume production we estimate it is from the mid-2000s) says that with too many false calls, operators’ discrimination drops during verification and real defects inevitably slip through; that in some plants operators loosen AOI parameters to reduce alarms instead of using the data to adjust the process; and that the data accumulated this way is polluted by false calls and nearly useless for statistical process control. This is one author’s observation, not a standard.

The fuse case figures also show that false calls were not a one-off batch. In the first trial, 143 units were flagged with 85.3% overcalls. In the second trial, 70 units were flagged with 91.4% overcalls, and the report speculated that unstable surface condition on the fuses affected the AOI judgment. In another run (the report heading says “3K sorting”), flagged units were measured with a meter and 2 real defects were found, for a 94.1% overcall rate (worked back from the rate and the 2 real defects: about 34 flagged, 32 of them false — our arithmetic; the report gives just the rate and the 2 defects). Two other card types, two lots each, had overcall rates of 90.5% and 84.2%, and 63.6% and 83.3%. Converting these seven figures, real defects made up roughly 6% to 36% of flagged units, mostly around 10%. The risk lies in a person having to find a few real defects among a mass of false signals. In the case, every flagged unit was checked under a microscope and its resistance measured with a meter, and AOI-2 and re-inspection were added to the process.

Q: What should re-inspection records include?

A: Location, type, result and photo — with the result in three categories. Our summary:

• Which lot, which board, which location, and what defect type AOI reported.
• The re-inspection result in three categories: real defect, false call, undetermined (do not count “undetermined” as a false call).
• The photo or screenshot number, and who did the re-inspection and when.
• For real defects, also record whether it was reworked and what AOI-2 found on the second look.

5. Why false calls happen, and whether the AOI itself is stable

Q: Why does AOI flag a solder joint that is fine?

A: Because it compares the color and shape of an image, not the solder joint itself. We have an internal AOI interpretation note on exactly this fuse non-wetting case: AOI checks solder climb (the fillet) by color comparison. In the images, a well-climbed joint shows a blue reflection and a non-wetted one a paler blue. The blue comes from reflection off the top surface of the solder; with non-wetting there is less blue, and too little blue gets flagged. In other words, anything that changes the reflected color can get a good joint flagged.

The same SMT007 article says solder joint images are inherently unstable and hard to reproduce, so AOI inspection of joints must balance catching defects against keeping false calls down, with the principle of “keep it simple.” This is the author’s advice, not a standard.

Q: If the same board is run again, will AOI give the same result?

A: The 2010 repeat tests showed the locations were almost exactly the same, but one set reported 1 extra point on the third run. We compared the four sets of test data point by point using “panel board number + component name.” In the first set, two boards were each run 10 times in a row (our reading: the same board run repeatedly); one reported 35 points all 10 times and the other 22 points all 10 times, with exactly the same locations every run. In the second set, two boards were each run 3 times: the first two runs reported exactly the same locations (118 and 80 points, matching the “known defect points” of 118 and 80 in the trial evaluation table, which records that AOI detected them all, testing 1 board each). On the third run, each reported 1 additional location beyond the same ones (119 and 81 points). We could not find whether that extra point was a real defect or a false call.

Why look at repeatability? You first need to be sure that rerunning the same board gives a consistent result before you can treat differences between boards as process variation rather than measurement variation. This is this article’s synthesis, not a conclusion in the records.

6. AOI did not flag it, but downstream found it: how to investigate escapes

Q: AOI did not report it and downstream found the defect. How do you investigate?

A: Look at it in two layers: how the defect arose, and why AOI did not see it. A 2014 weekly report recorded this: downstream reported two boards with reversed polarity. The report noted two points: the source was the tape and reel (the report says “from the tape”), and AOI could not detect it effectively because the pin 1 marking on the component was unclear. The report also has a sentence, “should be fine at machine loading,” that we did not fully understand, so we do not expand on it.

This example splits into two questions. First, at which station was the polarity reversed (orientation in the tape, loading, placement data)? Second, what does AOI rely on to judge polarity (is the marking on the component clearly visible)? Once the second layer is clear do you know whether to add a separate check. For the types of defects AOI cannot see, start with this article on our site: “Where to Place AOI: Before or After Reflow, 2D or 3D.”

There is one more detail in recording. IPC-9261A requires a defect to be recorded at the station where it is found, and states that this applies even if an earlier undetected defect caused the later one; it also requires defects to be assigned to the appropriate process step. This can be used directly: an escape is found downstream, so record it at the station where it was found, and add a column for “the station that should have stopped it.”

7. Step three: trace upstream, recording “where found” and “where caused” separately

Q: Once a defect is confirmed as real, how do you find which station caused it?

A: Classify the defect first, then rule out stations one at a time by defect type, changing one thing at a time. The SMT007 article’s view is that AOI pays off fastest when used as a feedback tool for improving yield; in reality many people just “inspect out” the defects, the data is polluted by false calls, and it is nearly useless for statistical process control. That is also why the first two steps (match the location, separate real from false) need to be done well first.

Figure: six steps of tracing back after an AOI defect call (illustration; the flow is this article’s synthesis; the basis for each step is in the sections of this article).

The fuse case is a complete example. It worked through these layers in order (all figures belong to that component and that line, not general specifications):

To turn AOI data into clues like these, three items on our 2009 improvement list to the placement machine maker pointed in this direction. One: AOI accumulates the top 1 to 5 defects each day and links them to the placement machine, so engineers know which feeder or nozzle has more pick errors or tombstoning. Two: yield is monitored with a moving average, and when it falls below 99.2% AOI flags it and attaches a summary table (which machine, which component, nozzle, feeder, top defect and defect percentage). Three: AOI compiles the problems and feeds them back automatically to the placement machine or printer for adjustment — to which the maker replied it would be “very hard to do.” We do not know whether it was ever built; we borrow it here just to show “where the data needs to connect.”

For which upstream station to check by defect type, continue with these articles on our site: for solder paste printing problems, “Troubleshooting Solder Paste Printing Defects”; for placement, “Troubleshooting Feeder Misfeeds and Jams” and “Nozzle Pick-Up Errors and Dropped Parts: What to Do”; for defects after reflow, “How to Investigate Reflow Soldering Defects.”

8. Before sending a report or accepting equipment, ask yourself 8 questions

9. When AOI flags defects, what to check first

Q: You have a batch of AOI defect calls. What is the first step?

A: Do not rush to change the reflow profile or the stencil; first sort out location and real versus false. Do these in order:

• See whether it appears across the whole board or repeatedly at the same location.
• Match the location: is the location marked on the board the same as on the AOI screen (see Section 3)?
• Re-inspect and record: real defect, false call, undetermined (see Section 4).
• Classify: are the same defect type and the same location concentrated?
• Go upstream: check printing, placement, reflow and materials by defect type (see Section 7).
• Change one thing, keep a comparison, and watch it for a while.

If you have not yet decided which station AOI belongs at, see “Where to Place AOI.” For which tests to pair with AOI, see “How to Combine PCBA Tests.”

Limitations of this article

Public sources: for IPC-7912A and IPC-9261A, we could read just the first 3 pages published on the website (cover, contents, overview and definitions). We did not read how “opportunities” are counted for each defect or the appendices, nor the defect clauses of IPC-A-610 or J-STD-001. The SMT007 article has no clear date and the page does not describe the author’s company’s business; its content is the author’s observation, not a standard. We also read another SMT007 article on closed-loop feedback between AOI and placement machines, but did not cite it, since its date and affiliation are equally unclear.

Internal records: machines, models and years (2009 to 2014) all differ, and the figures belong to those cases alone. The 1,269 points, and about 34 and 32 units, are our own back-calculations. The “wrong location marked” in the fuse case is the report’s inference, not confirmed. We did not confirm whether the AOI in the case had automatic marking. One sentence in the 2014 weekly report was not understood. We also do not know whether these cases recurred. The lists in Sections 3 and 4, the recording method in Section 6 and the 8 questions in Section 8 are this article’s synthesis, not taken from the records, and not a standard.

Frequently asked questions

What do AOI false calls and escapes mean? A false call is when AOI reports a problem that is not actually there, also called overcall or overkill. An escape is a real problem that AOI does not report, also called underkill. Usage varies between companies, so when you report a figure, say which one you mean and what the denominator is.

What AOI false call rate is normal? The public sources we read give no general threshold. Of our two old internal records, one had about 1,070 false calls per million inspection points, and in the other 85.3% of flagged units were false calls; the machines, products and methods differ, so they cannot be compared. A more practical approach is to state the denominator clearly, compare with downstream data or your own history, and check whether false calls are causing re-inspectors to miss real defects.

AOI flags many defects but repair keeps saying there is no problem. What should we do? Confirm the location first. The inference in the fuse case was that the marker on the board was placed on the wrong unit, so repair looked at a good unit and the bad one went unseen. Check the flagged location under a microscope first, then decide whether it is a false call; repair feedback should also say “where I looked and what I saw.”

Is AOI ppm the same as IPC DPMO? Not exactly. IPC-9261A defines DPMO as the number of defects divided by the number of defect opportunities, multiplied by one million, and explains that it is not strictly the same as PPM because “opportunities” are counted in a particular way in electronics assembly — for example, a component counts as one opportunity regardless of its number of leads. What denominator an AOI equipment document uses for ppm depends on how that document defines it.

Why does AOI judge a good solder joint as defective? Because it compares the color and shape of an image. Our internal note says AOI checks solder climb by color comparison: a well-climbed joint shows a blue reflection, while a non-wetted one is paler blue. Component surface condition, solder volume, board temperature, warpage and solder type can all affect the judgment. Criteria should be tuned to the product and process, with a record of each adjustment.

If the same board is rerun, will AOI give the same result? In the 2010 repeat tests, two boards were each run 10 times with the same locations reported every time; two others were each run 3 times, and on the third run each reported 1 extra location. So it is broadly stable, but not identical every time. Before using AOI data to look at process variation, confirm that rerunning the same board gives a consistent result.

Can AOI feed results back to the placement machine automatically? In concept, yes; in practice it depends on the equipment. In 2009 we asked a placement machine maker for automatic feedback, and the maker replied that it would be very hard to do; we do not know whether it was later built. An easier place to start (this is our suggestion) is to have AOI accumulate the top few defects each day, link them to feeders and nozzles, and let engineers adjust manually based on the data.

AOI missed a component with reversed polarity. How do we investigate? Investigate in two layers: at which station the polarity was reversed (orientation in the tape, loading, placement data), and why AOI did not see it. In the 2014 weekly report example, AOI could not detect it effectively because the pin 1 marking on the component was unclear, and the source was the tape and reel. Components with unclear markings need a separate check.

Working on a similar problem? Write to JIAHON and we will look at what support fits. We offer quick online consultations — one more person to talk it through with — and will do our part to help solve the problem.

Request a quote

Request a quote →