1

Duplicate Marking Utility for Large Datasets

I don't know at what point using cnt(select YourTable where YourField = CurrentField) > 1 becomes impractical, but I recently had the opportunity to work with a table that had 64,000+ records, thank you Wes, and that qualified for impractical. The select statement updates dynamically and that isn't necessary when you just want to find duplicates. This solution works best if you add 2 fields to the table to be marked and I named mine DStatus and MStatus for duplicates and multiple duplicates. Here is the code for a button...

let curField := first((select YourTable) order by YourField).YourField;
let curCount := 1;
let isDuplicate := false;
for r in (select YourTable) order by YourField do
if isDuplicate and r.YourField = curField then
r.(MStatus := true)
end;
if r.YourField = curField and (isDuplicate = true or curCount > 1) then
r.(DStatus := true);
curCount := 1;
isDuplicate := true
else
curCount := curCount + 1;
isDuplicate := false
end;
curField := r.YourField
end

Run and done. You filter on DStatus or MStatus to isolate the duplicate records and use a View layout element to view the individual duplicates. As far as I know, this only works in the Mac app, but it might also work in the iPad app. Using the Mac app the code took less than 45 seconds to process the records on my 2015 MacBook Pro.

I tried to use the code in the browser version and shut it down after nearly 20 minutes. I also tried using cnt(select YourTable where YourField = CurrentField) > 1 in the Mac app and gave up after 10 minutes of the spinning wheel and used Force Quit to shut the app down.

 

Screen Shot 2020-01-26 at 12.02.36 AM

 

Screen Shot 2020-01-26 at 12.03.08 AM

1 reply

null
    • 441349382
    • 5 mths ago
    • Reported - view

    That helped me thank you.

Content aside

  • 1 Likes
  • 5 mths agoLast active
  • 1Replies
  • 1207Views
  • 1 Following