{"id":71800,"date":"2020-09-21T14:17:02","date_gmt":"2020-09-21T11:17:02","guid":{"rendered":"https:\/\/en.buradabiliyorum.com\/why-twitters-image-cropping-algorithm-appears-to-have-white-bias\/"},"modified":"2020-09-21T14:17:02","modified_gmt":"2020-09-21T11:17:02","slug":"why-twitters-image-cropping-algorithm-appears-to-have-white-bias","status":"publish","type":"post","link":"https:\/\/buradabiliyorum.com\/en\/why-twitters-image-cropping-algorithm-appears-to-have-white-bias\/","title":{"rendered":"#Why Twitter\u2019s image cropping algorithm appears to have white bias"},"content":{"rendered":"

“#Why Twitter<\/a>\u2019s image cropping algorithm app<\/a>ears to have white bias<\/strong>”<\/p>\n

\n

If you want to watch Movies<\/a> or TV series visit the Dizi.BuradaBiliyorum.Com<\/a><\/span><\/strong><\/p>\n<\/blockquote>\n

<\/p>\n

\n
\n
\n
\n
\n
\n
\n
\nTwitter\u2018s algorithm for automatically cropping images attached to tweets often doesn\u2019t focus on the important content in them. A bother, for sure, but it seems like a minor one on the surface. However, over the weekend, researchers found that the cropping algorithm might have a more serious problem: white bias.<\/p>\n

Several users posted a lot of photos to show that in an image that has people with different colors,\u00a0Twitter chooses to show folks with lighter skin after cropping those images to fit its display parameters on its site and embeds<\/span>. Some of them even tried to reproduce results with fictional characters and dogs.<\/p>\n

If you tap on these images, you\u2019ll see an uncropped version of the image which includes more details such as other person or character. What\u2019s odd is that even if users flipped the order of where dark-skinned and light-skinned people appeared, the results were the same.\n<\/p><\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n

\n

Trying a horrible experiment\u2026<\/p>\n

Which will the Twitter algorithm pick: Mitch McConnell or Barack Obama? pic.twitter.com\/bR1GRyCkia<\/a><\/p>\n

\u2014 Tony \u201cAbolish (Pol)ICE\u201d Arcieri ? (@bascule) September 19, 2020<\/a>\n<\/p><\/blockquote>\n

\n

Happens with Michael Jackson too\u2026\u2026 pic.twitter.com\/foUMcExS2P<\/a><\/p>\n

\u2014 carter (@gnomestale) September 19, 2020<\/a>\n<\/p><\/blockquote>\n

\n

I wonder if Twitter does this to fictional characters too.<\/p>\n

Lenny Carl pic.twitter.com\/fmJMWkkYEf<\/a><\/p>\n

\u2014 Jordan Simonovski (@_jsimonovski) September 20, 2020<\/a>\n<\/p><\/blockquote>\n

\n

I tried it with dogs. Let’s see. pic.twitter.com\/xktmrNPtid<\/a><\/p>\n

\u2014 \u2013 M A R K \u2013 (@MarkEMarkAU) September 20, 2020<\/a>\n<\/p><\/blockquote>\n

\n

OMG! https:\/\/t.co\/5L9z473Hff<\/a> pic.twitter.com\/vVH6YO6GRv<\/a><\/p>\n

\u2014 BG ? #SouthernCollective (@joBeeGeorgeous) September 20, 2020<\/a>\n<\/p><\/blockquote>\n

\u00a0<\/p>\n

However, some people noted that there might be other factors than the color of the skin. And they who tried different methods found inconsistent results.<\/p>\n

\n

Does Twitter’s thumbnail-picker algorithm systematically prefer white faces over Black ones? <\/p>\n

I did an experiment. It’s not conclusive, but in my experiment with pictures of Barack Obama, Raphael Warnock, George W. Bush and Donald Trump, the hypothesized pattern didn’t appear. pic.twitter.com\/2ddcPR5CPi<\/a><\/p>\n

\u2014 Jeremy B. Merrill (@jeremybmerrill) September 20, 2020<\/a>\n<\/p><\/blockquote>\n

\n

testing pic.twitter.com\/xqHcOpz3Yg<\/a><\/p>\n

\u2014 garry (@garrynewman) September 20, 2020<\/a>\n<\/p><\/blockquote>\n

\n

More proof: pic.twitter.com\/CeCEOTsSJ8<\/a><\/p>\n

\u2014 Him Gajria (@himgajria) September 20, 2020<\/a>\n<\/p><\/blockquote>\n

\n

(Results update)
White-to-Black ratio: 40:52 (92 images)
Code used: https:\/\/t.co\/qkd9WpTxbK<\/a>
Final annotation: https:\/\/t.co\/OviLl80Eye<\/a>
(I’ve created @cropping_bias<\/a> to run the complete the experiment. Waiting for @Twitter<\/a> to approve Dev credentials) pic.twitter.com\/qN0APvUY5f<\/a><\/p>\n

\u2014 Vinay Prabhu (@vinayprabhu) September 20, 2020<\/a>\n<\/p><\/blockquote>\n

Twitter\u2019s Cheif Design Officer (CDO), Dantley Davis, said that the choice of cropping sometimes takes brightness of the background into consideration.<\/span><\/p>\n

\n

Here’s another example of what I’ve experimented with. It’s not a scientific test as it’s an isolated example, but it points to some variables that we need to look into. Both men now have the same suits and I covered their hands. We’re still investigating the NN. pic.twitter.com\/06BhFgDkyA<\/a><\/p>\n

\u2014 Dantley ?\u270a?? (@dantley) September 20, 2020<\/a>\n<\/p><\/blockquote>\n

In a thread, Binaca Kastl, a developer from Germany, explained that Twitter\u2019s algorithm might be cropping the image based on saliency<\/a> \u2014 an important point or part in an image that you\u2019re likely to look at first when you see it.<\/p>\n

\n

Probably Twitters Crop algorithm is a pretty simple Saliency. We will see\u2026 pic.twitter.com\/q4R0R8h3vh<\/a><\/p>\n

\u2014 Bianca Kastl (@bkastl) September 20, 2020<\/a>\n<\/p><\/blockquote>\n

Her theory is backed by Twitter\u2019s 2018 blog post that explained its neural network<\/a> built for image cropping. The post notes that earlier, the company took facial detection into account to crop images. However, that approach didn\u2019t work for images that didn\u2019t have a face in them. So the social network switched to a Saliency-based algorithm.<\/p>\n

[Read:\u00a0Are EVs too expensive? Here are 5 common myths, debunked]<\/em><\/p>\n

Even if Twitter\u2019s algorithm is not \u2018racist,\u2019 enough people have posted examples showing\u00a0the algorithm appears biased towards lighter skin tones, and the results are problematic.<\/span>. The company definitely needs to do some digging into their algorithm to understand the bias in its neural network. Anima Anandkumar,\u00a0Director of AI research at Nvidia, pointed out that the saliency algorithm might be trained using eye-tracking of straight male participants, and that would insert more bias into the algorithm.<\/p>\n

\n

Recording straight men where their eyes veer when they view female pictures is encoding objectification and sexualization of women in social media @Twitter<\/a> No one asks whose eyes are being tracked to record saliency. #ai<\/a> #bias<\/a> https:\/\/t.co\/coXwngSjiW<\/a><\/p>\n

\u2014 Prof. Anima Anandkumar (@AnimaAnandkumar) September 20, 2020<\/a>\n<\/p><\/blockquote>\n

Twitter spokesperson Liz Kelly tweeted that the firm tested the model and didn\u2019t find any bias. She added that the company will open-source its work for others to review and replicate. It might be possible that Twitter has ignored some factors while testing, and open-sourcing the study might help them find those blind spots.<\/p>\n

\n

thanks to everyone who raised this. we tested for bias before shipping the model and didn’t find evidence of racial or gender bias in our testing, but it\u2019s clear that we\u2019ve got more analysis to do. we’ll open source our work so others can review and replicate. https:\/\/t.co\/E6sZV3xboH<\/a><\/p>\n

\u2014 liz kelley (@lizkelley) September 20, 2020<\/a>\n<\/p><\/blockquote>\n

The company\u2019s Chief Technology<\/a> Officer (CTO), Parag Agarwal, said that the model needs continuous improvements and the team is eager to learn from this experience.<\/p>\n

\n

This is a very important question. To address it, we did analysis on our model when we shipped it, but needs continuous improvement. <\/p>\n

Love this public, open, and rigorous test \u2014 and eager to learn from this. https:\/\/t.co\/E8Y71qSLXa<\/a><\/p>\n

\u2014 Parag Agrawal (@paraga) September 20, 2020<\/a>\n<\/p><\/blockquote>\n

Light skin bias in algorithms is\u00a0well documented in fields ranging from\u00a0<\/span>healthcare<\/a>\u00a0to\u00a0<\/span>law enforcement.<\/span>\u00a0So large companies like Twitter need to continuously work on their systems to get rid of it. Plus, it needs to start an open dialog with the AI community to understand its blind spots.<\/p>\n

So you\u2019re interested in AI? Then <\/span>join our online event, TNW2020<\/span>, where you\u2019ll hear how artificial intelligence is transforming industries and businesses.<\/span><\/em><\/p>\n

\n Published September 21, 2020 \u2014 11:17 UTC\n <\/p>\n<\/p><\/div>\n