The Web Design Group

... Making the Web accessible to all.

Welcome Guest ( Log In | Register )

 
Reply to this topicStart new topic
> Update binary mask over image without redrawing everything
meteor
post Aug 5 2021, 11:11 AM
Post #1





Group: Members
Posts: 7
Joined: 5-August 21
Member No.: 28,031



Hi,

A few days ago, I've asked for help on an issue on Stack Overflow, but haven't received any replies. However, I still very much need help, so I figured I'd try asking a different community.

I'll link to the post instead of copying it since S/O has a feature to run code directly on the website, and the post includes a complete Minimum Working Example. Here it is: https://stackoverflow.com/questions/6865189...wing-everything

I'd be grateful if anyone can figure out how to do this efficiently -- I assume it has to be possible since I know it would be in something like java.
User is offlinePM
Go to the top of the page
Toggle Multi-post QuotingQuote Post
CharlesEF
post Aug 5 2021, 02:47 PM
Post #2


Programming Fanatic
********

Group: Members
Posts: 1,981
Joined: 27-April 13
From: Edinburg, Texas
Member No.: 19,088



I've never used 'canvas' so I can't help with that. But, have you looked into data grids? A data grid might fit your needs better than a 'canvas'.
User is offlinePM
Go to the top of the page
Toggle Multi-post QuotingQuote Post
Brian Chandler
post Aug 6 2021, 12:26 AM
Post #3


Jocular coder
********

Group: Members
Posts: 2,460
Joined: 31-August 06
Member No.: 43



Sorry, I don't know about canvas either. But first, you must be very very old. The word "wasteful" has been obsoletified; if it works, who cares how inefficient it is.

More constructively: it might be simpler to put the image there, then overlay the red-square grid as a separate layer - here is a question about this:
https://stackoverflow.com/questions/3008635...multiple-layers

Also remember the XOR trick: if you just want to switch an image between two states, and are not concerned which it is in, let the states be (images) A and B; let C = A xor B; then to switch, just XOR with C. HTH

User is offlinePM
Go to the top of the page
Toggle Multi-post QuotingQuote Post
meteor
post Aug 6 2021, 04:08 AM
Post #4





Group: Members
Posts: 7
Joined: 5-August 21
Member No.: 28,031



QUOTE
But first, you must be very very old. The word "wasteful" has been obsoletified; if it works, who cares how inefficient it is.


I care because it hurts my soul I think the user will care if it doesn't go smoothly. Which, in fact, it already doesn't when I run the file from my browser; there's a noticeable delay when clicking on the image.

Stacking two canvas objects on top of each other (that's what the stack overflow page you linked is doing) sounds like it could do the job, though. I'll try that.

EDIT: have not yet looked (nor heard of) data grids.

This post has been edited by meteor: Aug 6 2021, 05:05 AM
User is offlinePM
Go to the top of the page
Toggle Multi-post QuotingQuote Post
meteor
post Aug 6 2021, 05:02 AM
Post #5





Group: Members
Posts: 7
Joined: 5-August 21
Member No.: 28,031



Stacking canvas is working (thanks for the link). However, it doesn't seem to have solved the issue. I think redrawing the image wasn't the problem in the first place.

I now have another question. The real file actually requires access to 10.000 7x7 masks, on for each of 10.000 different images. The user is always shown one at random and has to 'guess' the correct mask. Point being, the information about the masks needs to be stored somewhere. What I've done right now is to just dump them into the html file, i.e, there is a declaration

CODE
var masks = [  {{10.000 7x7 boolean arrays}} ]


which I all typed up I'm kidding of course the html file is generated by python code

This means the html file has over 10.000 lines and it appears that this is slowing it down, given that the MWE is faster than the real file. Can anyone explain me why that is? Why would a long static declaration make execution slower? Is the code in the <script> tags compiled more than once?

I guess I'm going to generate 10.000 text files instead and read out the masks on demand, but I'd love to understand why there is a problem before I fix it.

The other thing I've noticed is that the program gets slower over time, and this is true for both the MWE and the proper file. I.e., if you spam click the image, it's fast at first and then gets slower and slower.
User is offlinePM
Go to the top of the page
Toggle Multi-post QuotingQuote Post
Brian Chandler
post Aug 6 2021, 09:53 AM
Post #6


Jocular coder
********

Group: Members
Posts: 2,460
Joined: 31-August 06
Member No.: 43



It's very hard to imagine what this is really all about. But if you show the user one image, and they have to guess the "correct" mask somehow, you only need the one particular mask available to check against. Or are you saying you actually *display* 10,000 masks?? Javascript is at the mercy of the user's computing environment, and I don't think sending arrays of 10,000 anything is generally a good idea.
User is offlinePM
Go to the top of the page
Toggle Multi-post QuotingQuote Post
meteor
post Aug 6 2021, 10:48 AM
Post #7





Group: Members
Posts: 7
Joined: 5-August 21
Member No.: 28,031



I only display one mask at a time. However, in the current version, the user can switch to a new image via keyboard click, which is randomly sampled from all available ones (or to be specific, all available ones that have a mask that's not all zeros). This new image will have its own mask.

This post has been edited by meteor: Aug 6 2021, 10:49 AM
User is offlinePM
Go to the top of the page
Toggle Multi-post QuotingQuote Post
Brian Chandler
post Aug 7 2021, 02:50 AM
Post #8


Jocular coder
********

Group: Members
Posts: 2,460
Joined: 31-August 06
Member No.: 43



QUOTE(meteor @ Aug 7 2021, 12:48 AM) *

I only display one mask at a time. However, in the current version, the user can switch to a new image via keyboard click, which is randomly sampled from all available ones (or to be specific, all available ones that have a mask that's not all zeros). This new image will have its own mask.


Well, I think "Ajax" is the answer. The click to change to a new image simply fetches the necessary data from the server. You don't surely mean that currently you are loading 10,000 images?

Alternatively, choose an arbitrary maximum number of images in a session - say 500, and just send the mask data for a preselected (pseudo-)random set of images. Each mask is 49 bits of data, so you can easily encode it in 16 hex characters.

Can we see the actual site? What on earth is it for?
User is offlinePM
Go to the top of the page
Toggle Multi-post QuotingQuote Post
meteor
post Aug 7 2021, 02:13 PM
Post #9





Group: Members
Posts: 7
Joined: 5-August 21
Member No.: 28,031



Apologies; I'm sure this is something I should have said immediately. I'm running the html as a local file as of now, so there is no site.

QUOTE
What on earth is it for?


If you're familiar with Machine Learning, the short answer is it lets you guess thresholded neuron activations of the penultimate layer of resnet trained on ade20k.

If you're not, I'll take a shot at a simple explanation (but read at your own risk of wasting time on something you'll never need again)

There are these things called convolutional neural nets (CNNs) that are used to process images. An image goes in, and something goes out, like a label of what the image is about. To do this, the CNN iteratively transforms the data to have more channels, be lower resolution, and store higher-level information. E.g., say it's a 200x200 pixel image. The initial input are the three 200x200 color channels. That means we initially have 3 channels, a very high resolution, and each bit of information is as low level as it gets, i.e., the brightness of one pixel. It may now transform this into, say, 32 100x100 'images' which I'll now call filters. In each of those filters, each 'pixel' which I'll now call cell corresponds to a 2-by-2 pixel area of the initial image, but rather than encoding brightness, it encodes some simple concept like 'horizontal line'. I.e., one of the 100x100 filters may be such that each cell has a high number the corresponding image area looks like it has a horizontal line. Then another filter may have cells detecting vertical lines, another areas with green colors, and so on. Together, the 32 100x100 filters should encode all relevant information from the initial image.

This process is then repeated, so in the next step, we may have 64 50x50 filters. Fast forward, the penultimate layer has exactly 512 7x7 filters. At this point, the filters reacts to high-level properties, like 'tree' or 'person' or 'shelf', i.e., each cell in them has a high number if the CNN thinks the corresponding area in the original image contains a tree/person/shelf. But the truth is that no-one really understands what the cells activate for, and that's what my project is trying to help with. For just one of the 512 filters, one gets to guess on what parts of an image the filter activates. I.e., one sees the image, the 7x7 grid, and gets to draw a mask in it. Then by keyboard click, one sees the 'real' mask, which is just a thresholded version of the filter, i.e., all cells with a number above a certain value count as being on. Then, one gets another image for the same filter and can guess again. The hope is that it helps to better understand what the filter is doing.

The first filter in this CNN seems to primarily detect work surfaces in kitchens. Here's an example of an image where it behaves mostly as expected, although even here it's hard to understand why e.g. it activated on the two bottom right cells of the blue area. There are many cases that are much more perplexing, like where the same filter activates on part of a road, and it's hard to tell whether the CNN made a mistake or has an ontology where roads and work surfaces are similar.
User is offlinePM
Go to the top of the page
Toggle Multi-post QuotingQuote Post
Brian Chandler
post Aug 8 2021, 01:51 PM
Post #10


Jocular coder
********

Group: Members
Posts: 2,460
Joined: 31-August 06
Member No.: 43



Oh crumbs, it's something interesting(!)

Anyway, I would just reload the page for each image. Put all the javascript in a separate file, then the loading should only be a very simple html page plus the new image.
User is offlinePM
Go to the top of the page
Toggle Multi-post QuotingQuote Post
meteor
post Aug 9 2021, 04:57 AM
Post #11





Group: Members
Posts: 7
Joined: 5-August 21
Member No.: 28,031



Actually, that's probably a really good solution. Right now I've been hitting refresh every couple of images manually, and that made it usable.

QUOTE
Put all the javascript in a separate file,


I totally forgot that you can do that. Also a good idea.
User is offlinePM
Go to the top of the page
Toggle Multi-post QuotingQuote Post
meteor
post Aug 9 2021, 05:48 AM
Post #12





Group: Members
Posts: 7
Joined: 5-August 21
Member No.: 28,031



Yeah, this is working great. You barely even notice that it's refreshing the page instead of just changing the image. Thanks :-)
User is offlinePM
Go to the top of the page
Toggle Multi-post QuotingQuote Post

Reply to this topicStart new topic
2 User(s) are reading this topic (2 Guests and 0 Anonymous Users)
0 Members:

 



- Lo-Fi Version Time is now: 18th March 2024 - 10:41 PM