python (12.9k questions)

javascript (9.2k questions)

reactjs (4.7k questions)

java (4.2k questions)

java (4.2k questions)

c# (3.5k questions)

c# (3.5k questions)

html (3.3k questions)

Questions - self-attention

Swin Transformer attention maps visualization

I am using a Swin Transformer for a hierarchical problem of multi calss multi label classification. I would like to visualize the self attention maps on my input image trying to extract them from the ...
test-img

Imanuel Rozenberg

maps

visualization

transformer-model

explain

self-attention

Votes: 0

Answers: 1

Latest Answer

I am also researching the same, while I don't have anything specific to SWIN. Here are some resources related to Vision Transformers. I hope it helps: https://jacobgil.github.io/deeplearning/vision-tr...
test-img

Aakash Gupta

I want to ask you about the structure of "query, key, value" of "transformer"

I'm a beginner at NLP. So I'm trying to reproduce the most basic transformer all you need code. But I got a question while doing it. In the MultiHeadAttention layer, I printed out the shape of "q...
test-img

user16579274

nlp

pytorch

translation

transformer-model

self-attention

Votes: 0

Answers: 0

For an image or sequence, what is the properties transformers use?

Today my teacher ask me a question: he said the CNN is use the translation invariance of the images or matrixs. So what is the properties Transformer uses ???
test-img

qicheng wang

conv-neural-network

transformer-model

self-attention

Votes: 0

Answers: 1

Latest Answer

There are two main properties of transformers that makes them so appealing compared to convolutions: A transformer is permutation equivariant. This makes transformers very useful for set predictions....
test-img

Shai

Posts

Questions

Blogs

Jobs